Compare commits
134 Commits
bcbe0b5e8c
..
v3.0.0
| Author | SHA1 | Date | |
|---|---|---|---|
| 528b7004e3 | |||
| 41c4667eb4 | |||
| c3ed36f178 | |||
| 93e55e8e37 | |||
| 3f12b60261 | |||
| 6d13fa425a | |||
| a0a987d561 | |||
| 3e04da3926 | |||
| 87d3c27b65 | |||
| 36546c46c9 | |||
| ac27ea3176 | |||
| eb3cf72969 | |||
| 4b6db84afc | |||
| 28f5b5d150 | |||
| 4883820f25 | |||
| a6a46af12e | |||
| bf7fa49a16 | |||
| 76d9c35363 | |||
| 71cf79eeaa | |||
| a16accf443 | |||
| 78b5e86dd6 | |||
| 281358ff04 | |||
| d8cccbb9e0 | |||
| d7bfe6b596 | |||
| e5a731eb29 | |||
| 6fe6e4cd55 | |||
| 6834b916cd | |||
| 745fe38e63 | |||
| 43c5c12fb9 | |||
| c9b7f0701b | |||
| b8d08925a2 | |||
| dc40729646 | |||
| 432b2d5b1e | |||
| 47d9cae15b | |||
| 595eb5e06a | |||
| 19bdd3faa9 | |||
| 16e68236a0 | |||
| dcbe640fae | |||
| 96a903aaff | |||
| 6e3e6a4194 | |||
| 9216a8687f | |||
| 88849cc16b | |||
| 58cea88f52 | |||
| b25085bcd7 | |||
| cf6c312e04 | |||
| da4ce37aff | |||
| 70d12615b8 | |||
| 97d6daaa13 | |||
| 95db88388b | |||
| a864ca4f7b | |||
| 5bb53d7f9d | |||
| 48e901cac1 | |||
| 4d11beb296 | |||
| 57192a49f9 | |||
| 9937984a5a | |||
| 6fba04cd00 | |||
| 35b1b2e6d5 | |||
| 98de298e5b | |||
| b470782e97 | |||
| 5a05bb86e1 | |||
| 99668ec0d8 | |||
| 88e07f0723 | |||
| 384386a34e | |||
| e694f17be9 | |||
| 2d2a42d523 | |||
| a371045b41 | |||
| da867406e2 | |||
| 21c7aa31ea | |||
| 4def59c86d | |||
| d345839634 | |||
| 7d0ea4e388 | |||
| 38572b1c80 | |||
| fedfa8df3a | |||
| cb1588ecc0 | |||
| 73ae8a046f | |||
| 6a3bb9ff51 | |||
| ca3bca59e7 | |||
| c8bd8512ce | |||
| 129cc59793 | |||
| 5571dfa46d | |||
| 52c62b5c8d | |||
| 0b0f797fae | |||
| 48ea1fe720 | |||
| af9f4c77db | |||
| 76dcb8721a | |||
| cefa33c00d | |||
| 6e5c4dc623 | |||
| 1b04d67e20 | |||
| bce79867e4 | |||
| e39cda9ec4 | |||
| 94d100675e | |||
| 19ec96c544 | |||
| 49e68cc263 | |||
| 3a0dbc9ba7 | |||
| 9a20cc1c84 | |||
| 49de9fe42b | |||
| 2b39af8c1c | |||
| 05af819b3e | |||
| e93240ff79 | |||
| 370dfd95bc | |||
| e97b7282ff | |||
| 9a14c50250 | |||
| e7d24f0677 | |||
| d357ae3156 | |||
| 03936243e4 | |||
| a78fc5ba9a | |||
| 8f213e780c | |||
| 88a8cac048 | |||
| aae0c587e8 | |||
| f73f954b1a | |||
| 76942721ad | |||
| 437b66e73a | |||
| fce71bb946 | |||
| 69650b6464 | |||
| 0e7890a465 | |||
| f5f7db6c2a | |||
| aab62cbe41 | |||
| 7a0c57a79c | |||
| e8479ee4fb | |||
| 1f3bc2acf4 | |||
| 5968a7b593 | |||
| e801eec47d | |||
| dc1af8c585 | |||
| ffb8848a96 | |||
| fa491487b7 | |||
| 4f8d6a41f9 | |||
| 18f1dbc052 | |||
| a1ec25a441 | |||
| 89ba6bae82 | |||
| 2af3c481bb | |||
| 00f63697b0 | |||
| 2ef055a41a | |||
| ec96fced40 | |||
| 5b973cf534 |
@@ -0,0 +1,33 @@
|
|||||||
|
# Keep the build context small and reproducible: never ship local state,
|
||||||
|
# dependencies, or build outputs — they are rebuilt inside the image.
|
||||||
|
|
||||||
|
# VCS / tooling
|
||||||
|
.git
|
||||||
|
.gitignore
|
||||||
|
**/.DS_Store
|
||||||
|
|
||||||
|
# Compose file — used to build/run, not needed inside the image context
|
||||||
|
docker-compose.yml
|
||||||
|
docker-compose.*.yml
|
||||||
|
|
||||||
|
# Secrets and local env
|
||||||
|
.env
|
||||||
|
.env.*
|
||||||
|
!.env.example
|
||||||
|
|
||||||
|
# Node
|
||||||
|
frontend/node_modules
|
||||||
|
frontend/.svelte-kit
|
||||||
|
frontend/build
|
||||||
|
frontend/.vite
|
||||||
|
|
||||||
|
# Go
|
||||||
|
backend/server
|
||||||
|
backend/**/*.test
|
||||||
|
|
||||||
|
# Docs / reference (not needed to build the image)
|
||||||
|
docs/reference
|
||||||
|
|
||||||
|
# Editor / OS
|
||||||
|
.vscode
|
||||||
|
.idea
|
||||||
@@ -1,16 +1,83 @@
|
|||||||
# =============================================================================
|
# =============================================================================
|
||||||
# Tanabata File Manager — environment variables
|
# Tanabata File Manager — environment variables
|
||||||
# Copy to .env and fill in the values.
|
#
|
||||||
|
# Copy to .env and fill in the secrets:
|
||||||
|
# cp .env.example .env
|
||||||
|
# docker compose up -d --build
|
||||||
# =============================================================================
|
# =============================================================================
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Docker Compose (read by the compose CLI, ignored by the app)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Profiles to enable. "with-db" runs the bundled Postgres container. Leave
|
||||||
|
# EMPTY to skip it and use a Postgres running on the host instead — then point
|
||||||
|
# DATABASE_URL at host.docker.internal (see the Database section below).
|
||||||
|
COMPOSE_PROFILES=with-db
|
||||||
|
|
||||||
|
# Host port the app is published on, bound to 127.0.0.1 (loopback) — a reverse
|
||||||
|
# proxy on the host fronts it (see README → Reverse proxy). The container always
|
||||||
|
# listens on 42776. To expose the app directly without a proxy, drop the
|
||||||
|
# "127.0.0.1:" prefix on the ports line in docker-compose.yml.
|
||||||
|
APP_PORT=42776
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Volume mounts (Docker Compose; ignored by the app)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# By default the app's data and the database live in named Docker volumes
|
||||||
|
# (app_files, app_thumbs, app_import, db_data). To keep them in specific folders
|
||||||
|
# on the host instead, point any of these at a host path — absolute, or relative
|
||||||
|
# to this file (e.g. ./data/files). Unset = named volume.
|
||||||
|
# FILES_DIR=/var/lib/tanabata/files
|
||||||
|
# THUMBS_DIR=/var/lib/tanabata/thumbs
|
||||||
|
# IMPORT_DIR=/var/lib/tanabata/import
|
||||||
|
# DB_DIR=/var/lib/tanabata/db
|
||||||
|
|
||||||
|
# When bind-mounting the app folders above, the container must be able to write
|
||||||
|
# to them. Set PUID/PGID to the owner of those folders and create them with
|
||||||
|
# matching ownership first, e.g.:
|
||||||
|
# sudo mkdir -p /var/lib/tanabata/{files,thumbs,import}
|
||||||
|
# sudo chown -R 1000:1000 /var/lib/tanabata
|
||||||
|
# PUID=1000
|
||||||
|
# PGID=1000
|
||||||
|
# Defaults match the image's tanabata user (42776), which owns the named volumes. The
|
||||||
|
# DB folder is handled by Postgres itself and needs no PUID/PGID.
|
||||||
|
# PUID=42776
|
||||||
|
# PGID=42776
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Server
|
# Server
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
LISTEN_ADDR=:8080
|
# 42776 is the project's default port: the sum of the Unicode code points of
|
||||||
|
# 七夕 (七 U+4E03 = 19971, 夕 U+5915 = 22805).
|
||||||
|
LISTEN_ADDR=:42776
|
||||||
JWT_SECRET=change-me-to-a-random-32-byte-secret
|
JWT_SECRET=change-me-to-a-random-32-byte-secret
|
||||||
JWT_ACCESS_TTL=15m
|
JWT_ACCESS_TTL=15m
|
||||||
JWT_REFRESH_TTL=720h
|
JWT_REFRESH_TTL=720h
|
||||||
|
|
||||||
|
# How long a content token is valid. It's a single-file capability the client
|
||||||
|
# puts in a media URL to open/stream an original by link (e.g. a long video in a
|
||||||
|
# new tab), so playback survives the short access-token expiry and session
|
||||||
|
# rotation. Longer = fewer interruptions but a wider window in which a leaked URL
|
||||||
|
# can read that one file; it can't be revoked before expiry. Keep it roughly as
|
||||||
|
# long as a viewing session lasts.
|
||||||
|
CONTENT_TOKEN_TTL=6h
|
||||||
|
|
||||||
|
# How long a graceful shutdown waits for in-flight requests to finish after the
|
||||||
|
# app receives SIGTERM/SIGINT (e.g. on `docker compose up --build`, which
|
||||||
|
# recreates the container). This SAME value is fed to the container's
|
||||||
|
# `stop_grace_period` in docker-compose.yml, so Docker won't SIGKILL the app
|
||||||
|
# mid-drain — set it in one place here and both stay in sync. A single upload or
|
||||||
|
# video stream that outlasts this window is still cut; raise it if you routinely
|
||||||
|
# move very large files. Accepts Go/Compose durations (s, m, h).
|
||||||
|
SHUTDOWN_TIMEOUT=15s
|
||||||
|
|
||||||
|
# Reverse-proxy hops (comma-separated CIDRs/IPs) whose X-Forwarded-For is trusted,
|
||||||
|
# so the auth rate limiter sees real client IPs instead of the proxy's. The default
|
||||||
|
# covers loopback and the Docker bridge ranges a host nginx reaches the container
|
||||||
|
# through; widen/narrow it to match your proxy. Leave at the default for the
|
||||||
|
# standard "host nginx → 127.0.0.1" setup.
|
||||||
|
TRUSTED_PROXIES=127.0.0.1/32,::1/128,172.16.0.0/12
|
||||||
|
|
||||||
# Initial administrator, created on first startup if it does not yet exist.
|
# Initial administrator, created on first startup if it does not yet exist.
|
||||||
# Changing the password later (via the API) is preserved across restarts.
|
# Changing the password later (via the API) is preserved across restarts.
|
||||||
ADMIN_USERNAME=admin
|
ADMIN_USERNAME=admin
|
||||||
@@ -19,10 +86,25 @@ ADMIN_PASSWORD=change-me-before-first-run
|
|||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Database
|
# Database
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
DATABASE_URL=postgres://tanabata:password@localhost:5432/tanabata?sslmode=disable
|
# Credentials for the bundled Postgres container (the "with-db" profile).
|
||||||
|
# Keep these in sync with DATABASE_URL below.
|
||||||
|
POSTGRES_DB=tanabata
|
||||||
|
POSTGRES_USER=tanabata
|
||||||
|
POSTGRES_PASSWORD=password
|
||||||
|
|
||||||
|
# Connection string the app uses. Pick ONE to match your database mode:
|
||||||
|
#
|
||||||
|
# • Bundled container DB (COMPOSE_PROFILES=with-db) — host is the "db" service:
|
||||||
|
DATABASE_URL=postgres://tanabata:password@db:5432/tanabata?sslmode=disable
|
||||||
|
#
|
||||||
|
# • Postgres on the host (COMPOSE_PROFILES empty):
|
||||||
|
# DATABASE_URL=postgres://tanabata:password@host.docker.internal:5432/tanabata?sslmode=disable
|
||||||
|
#
|
||||||
|
# • Bare-metal `go run` (no Docker):
|
||||||
|
# DATABASE_URL=postgres://tanabata:password@localhost:5432/tanabata?sslmode=disable
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Storage
|
# Storage (paths inside the container; backed by named volumes in compose)
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
FILES_PATH=/data/files
|
FILES_PATH=/data/files
|
||||||
THUMBS_CACHE_PATH=/data/thumbs
|
THUMBS_CACHE_PATH=/data/thumbs
|
||||||
@@ -37,8 +119,40 @@ THUMB_WIDTH=160
|
|||||||
THUMB_HEIGHT=160
|
THUMB_HEIGHT=160
|
||||||
PREVIEW_WIDTH=1920
|
PREVIEW_WIDTH=1920
|
||||||
PREVIEW_HEIGHT=1080
|
PREVIEW_HEIGHT=1080
|
||||||
|
# Pixel cap (width×height) for the pure-Go fallback decoder, used only when
|
||||||
|
# vipsthumbnail is NOT installed; larger images then get a placeholder. With vips
|
||||||
|
# present (the default image) thumbnails shrink on load, so this limit — and its
|
||||||
|
# RAM cost — don't apply. Also bounds a decompression bomb. Default ~300 Mpx.
|
||||||
|
THUMB_MAX_PIXELS=300000000
|
||||||
|
# How many thumbnails/previews may be generated at once. Each resize already uses
|
||||||
|
# every core, so a burst of large images otherwise pegs the CPU and RAM. 0 = auto
|
||||||
|
# (half the available CPUs).
|
||||||
|
THUMB_CONCURRENCY=0
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Import
|
# Import
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
IMPORT_PATH=/data/import
|
IMPORT_PATH=/data/import
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Duplicate detection
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Maximum perceptual-hash distance (Hamming, out of 64 bits) for two files to be
|
||||||
|
# treated as duplicate candidates. Lower = stricter (fewer, more confident
|
||||||
|
# matches); higher = looser (catches more re-encodes/resizes but risks false
|
||||||
|
# positives). On real libraries the distance histogram climbs steeply in the 8–10
|
||||||
|
# band — coincidental "vaguely similar" pairs, not duplicates — so 4 keeps the
|
||||||
|
# genuine-duplicate signal without that noise (and far fewer pairs to cluster).
|
||||||
|
# Used only by the dedup tool's pairs rebuild — see the dedup CLI /
|
||||||
|
# `docker compose run --rm dedup`. Code default is 10.
|
||||||
|
DUPLICATE_HASH_THRESHOLD=4
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Static SPA
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Leave UNSET here. The Docker image already serves the built SPA from
|
||||||
|
# /app/static and compose pins STATIC_DIR for the container — an empty value in
|
||||||
|
# .env would be injected into the container and disable SPA serving. Set this
|
||||||
|
# only for a bare-metal deploy where the Go server serves a built SPA; leave it
|
||||||
|
# unset in local dev, where the Vite dev server serves the UI.
|
||||||
|
# STATIC_DIR=/path/to/frontend/build
|
||||||
|
|||||||
@@ -0,0 +1,87 @@
|
|||||||
|
name: deploy
|
||||||
|
|
||||||
|
# Build the image and (re)start the compose stack on the production host
|
||||||
|
# whenever master moves. Also runnable manually from the Gitea Actions tab.
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
branches: [master]
|
||||||
|
workflow_dispatch: {}
|
||||||
|
|
||||||
|
# One deploy at a time; queue rather than cancel an in-flight run.
|
||||||
|
concurrency:
|
||||||
|
group: deploy-prod
|
||||||
|
cancel-in-progress: false
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
deploy:
|
||||||
|
# Self-hosted act_runner registered on the prod host with the "host" label
|
||||||
|
# (shell executor), so the job uses the host's git + Docker daemon and the
|
||||||
|
# existing clone in /opt/tanabata. See docs/DEPLOY.md for runner setup.
|
||||||
|
#
|
||||||
|
# Only shell steps here (no `uses:` actions), so the host needs git + docker
|
||||||
|
# and nothing else — no node, no go, no rsync. The test step below runs the
|
||||||
|
# Go and Node toolchains inside throwaway containers, so nothing has to be
|
||||||
|
# installed on the host.
|
||||||
|
runs-on: host
|
||||||
|
|
||||||
|
env:
|
||||||
|
DEPLOY_DIR: /opt/tanabata
|
||||||
|
|
||||||
|
steps:
|
||||||
|
- name: Pull latest master
|
||||||
|
# DEPLOY_DIR is a git clone set up once at deploy time. reset --hard
|
||||||
|
# makes it match origin exactly; .env is untracked (.gitignore) so it
|
||||||
|
# is never touched.
|
||||||
|
run: |
|
||||||
|
cd "$DEPLOY_DIR"
|
||||||
|
git fetch --prune origin
|
||||||
|
git reset --hard origin/master
|
||||||
|
|
||||||
|
- name: Run tests
|
||||||
|
working-directory: /opt/tanabata
|
||||||
|
# Everything runs INSIDE throwaway toolchain containers — the host only
|
||||||
|
# needs docker (which it already uses for `docker compose`). Nothing (Go,
|
||||||
|
# Node, Postgres, vips/ffmpeg/exiftool) has to be installed on the host.
|
||||||
|
# The orchestration below uses only bash builtins + docker. A failure here
|
||||||
|
# fails the job, so a red build never reaches production. Module/npm caches
|
||||||
|
# persist in named volumes for speed.
|
||||||
|
shell: bash
|
||||||
|
run: |
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
docker rm -f tfm-ci-db >/dev/null 2>&1 || true
|
||||||
|
docker network rm tfm-ci-net >/dev/null 2>&1 || true
|
||||||
|
docker network create tfm-ci-net >/dev/null
|
||||||
|
trap 'docker rm -f tfm-ci-db >/dev/null 2>&1 || true; docker network rm tfm-ci-net >/dev/null 2>&1 || true' EXIT
|
||||||
|
|
||||||
|
docker run -d --name tfm-ci-db --network tfm-ci-net \
|
||||||
|
-e POSTGRES_PASSWORD=postgres postgres:14-alpine >/dev/null
|
||||||
|
# Wait for Postgres, using pg_isready inside the DB container (no host tools).
|
||||||
|
for ((i = 0; i < 30; i++)); do
|
||||||
|
docker exec tfm-ci-db pg_isready -U postgres >/dev/null 2>&1 && break
|
||||||
|
sleep 1
|
||||||
|
done
|
||||||
|
|
||||||
|
# Backend: full suite, including the integration tests, against the
|
||||||
|
# ephemeral Postgres. -buildvcs=false since the .git dir isn't mounted.
|
||||||
|
docker run --rm --network tfm-ci-net \
|
||||||
|
-v /opt/tanabata/backend:/src -w /src \
|
||||||
|
-v tfm-ci-gomod:/go/pkg/mod -v tfm-ci-gocache:/root/.cache/go-build \
|
||||||
|
-e CGO_ENABLED=0 \
|
||||||
|
-e TANABATA_TEST_ADMIN_DSN="postgres://postgres:postgres@tfm-ci-db:5432/postgres?sslmode=disable" \
|
||||||
|
golang:1.26-alpine go test -buildvcs=false -count=1 ./...
|
||||||
|
|
||||||
|
# Frontend: type-check + production build (also validates openapi via
|
||||||
|
# the generate:types prestep).
|
||||||
|
docker run --rm \
|
||||||
|
-v /opt/tanabata:/repo -w /repo/frontend \
|
||||||
|
-v tfm-ci-npm:/root/.npm \
|
||||||
|
node:22-alpine sh -c "npm ci && npm run check && npm run build"
|
||||||
|
|
||||||
|
- name: Build image and start the stack
|
||||||
|
working-directory: /opt/tanabata
|
||||||
|
# .env must already exist in DEPLOY_DIR on the host (secrets + DB mode).
|
||||||
|
run: docker compose up -d --build --remove-orphans
|
||||||
|
|
||||||
|
- name: Prune dangling build layers
|
||||||
|
run: docker image prune -f
|
||||||
@@ -13,16 +13,18 @@ Monorepo: `backend/` (Go) + `frontend/` (SvelteKit).
|
|||||||
|
|
||||||
## Key documents (read before coding)
|
## Key documents (read before coding)
|
||||||
|
|
||||||
- `openapi.yaml` — full REST API specification (36 paths, 58 operations)
|
- `openapi.yaml` — full REST API specification (44 paths, 67 operations)
|
||||||
|
- `docs/REQUIREMENTS.md` — product requirements (functional + non-functional)
|
||||||
|
- `docs/ARCHITECTURE.md` — system architecture overview (layers, data flow, decisions)
|
||||||
- `docs/GO_PROJECT_STRUCTURE.md` — backend architecture, layer rules, DI pattern
|
- `docs/GO_PROJECT_STRUCTURE.md` — backend architecture, layer rules, DI pattern
|
||||||
- `docs/FRONTEND_STRUCTURE.md` — frontend architecture, CSS approach, API client
|
- `docs/FRONTEND_STRUCTURE.md` — frontend architecture, CSS approach, API client
|
||||||
- `docs/Описание.md` — product requirements in Russian
|
- `backend/migrations/` — database schema as goose migrations (4 schemas, 19 tables)
|
||||||
- `backend/migrations/001_init.sql` — database schema (4 schemas, 16 tables)
|
|
||||||
|
|
||||||
## Design reference
|
## Design reference
|
||||||
|
|
||||||
The `docs/reference/` directory contains the previous Python/Flask version.
|
Visual design tokens for the frontend (carried over from the previous
|
||||||
Use its visual design as the basis for the new frontend:
|
Python/Flask version):
|
||||||
|
|
||||||
- Color palette: #312F45 (bg), #9592B5 (accent), #444455 (tag default), #111118 (elevated)
|
- Color palette: #312F45 (bg), #9592B5 (accent), #444455 (tag default), #111118 (elevated)
|
||||||
- Font: Epilogue (variable weight)
|
- Font: Epilogue (variable weight)
|
||||||
- Dark theme is primary
|
- Dark theme is primary
|
||||||
@@ -32,6 +34,7 @@ Use its visual design as the basis for the new frontend:
|
|||||||
- Floating selection bar for multi-select
|
- Floating selection bar for multi-select
|
||||||
|
|
||||||
## Backend commands
|
## Backend commands
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cd backend
|
cd backend
|
||||||
go run ./cmd/server # run dev server
|
go run ./cmd/server # run dev server
|
||||||
@@ -39,6 +42,7 @@ go test ./... # run all tests
|
|||||||
```
|
```
|
||||||
|
|
||||||
## Frontend commands
|
## Frontend commands
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cd frontend
|
cd frontend
|
||||||
npm run dev # vite dev server
|
npm run dev # vite dev server
|
||||||
@@ -55,4 +59,4 @@ npm run generate:types # regenerate API types from openapi.yaml
|
|||||||
- Git: conventional commits with scope — `type(scope): message`
|
- Git: conventional commits with scope — `type(scope): message`
|
||||||
- `(backend)` for Go backend code
|
- `(backend)` for Go backend code
|
||||||
- `(frontend)` for SvelteKit/TypeScript code
|
- `(frontend)` for SvelteKit/TypeScript code
|
||||||
- `(project)` for root-level files (.gitignore, docs/reference, structure)
|
- `(project)` for root-level files (.gitignore, docs, structure)
|
||||||
|
|||||||
@@ -0,0 +1,96 @@
|
|||||||
|
# =============================================================================
|
||||||
|
# Tanabata File Manager — single-image build
|
||||||
|
#
|
||||||
|
# Produces one container that serves the SvelteKit SPA (built to static files)
|
||||||
|
# and the Go API on the same port. There is no Node runtime in the final image:
|
||||||
|
# the frontend uses adapter-static, so stage 1 emits plain HTML/CSS/JS that the
|
||||||
|
# Go binary serves directly (see STATIC_DIR).
|
||||||
|
# =============================================================================
|
||||||
|
|
||||||
|
# -----------------------------------------------------------------------------
|
||||||
|
# Stage 1 — build the frontend (static SPA)
|
||||||
|
# -----------------------------------------------------------------------------
|
||||||
|
FROM node:22-alpine AS frontend
|
||||||
|
|
||||||
|
WORKDIR /src/frontend
|
||||||
|
|
||||||
|
# Install dependencies first so this layer is cached unless the lockfile changes.
|
||||||
|
COPY frontend/package.json frontend/package-lock.json ./
|
||||||
|
RUN npm ci
|
||||||
|
|
||||||
|
# `npm run build` runs `generate:types`, which reads ../openapi.yaml relative to
|
||||||
|
# the frontend directory — place the spec one level up to match the repo layout.
|
||||||
|
COPY openapi.yaml /src/openapi.yaml
|
||||||
|
COPY frontend/ ./
|
||||||
|
|
||||||
|
RUN npm run build
|
||||||
|
# Output: /src/frontend/build (index.html, _app/, fonts, service-worker.js, …)
|
||||||
|
|
||||||
|
# -----------------------------------------------------------------------------
|
||||||
|
# Stage 2 — build the Go server (static binary)
|
||||||
|
# -----------------------------------------------------------------------------
|
||||||
|
FROM golang:1.26-alpine AS backend
|
||||||
|
|
||||||
|
WORKDIR /src/backend
|
||||||
|
|
||||||
|
# Download modules first so this layer is cached unless go.mod/go.sum changes.
|
||||||
|
COPY backend/go.mod backend/go.sum ./
|
||||||
|
RUN go mod download
|
||||||
|
|
||||||
|
COPY backend/ ./
|
||||||
|
|
||||||
|
# CGO is disabled: the binary shells out to external tools at runtime
|
||||||
|
# (vipsthumbnail for image thumbnails, ffmpeg for video frames, exiftool for
|
||||||
|
# metadata) and falls back to pure-Go image processing (disintegration/imaging)
|
||||||
|
# when vips is absent, so it stays fully static and portable across base images.
|
||||||
|
RUN CGO_ENABLED=0 GOOS=linux go build -trimpath -ldflags="-s -w" -o /out/server ./cmd/server
|
||||||
|
# dedup: offline maintenance CLI for duplicate detection (hash backfill + pairs
|
||||||
|
# rescan). Shipped alongside the server so it can be run with `docker exec`.
|
||||||
|
RUN CGO_ENABLED=0 GOOS=linux go build -trimpath -ldflags="-s -w" -o /out/dedup ./cmd/dedup
|
||||||
|
|
||||||
|
# -----------------------------------------------------------------------------
|
||||||
|
# Stage 3 — minimal runtime
|
||||||
|
#
|
||||||
|
# Alpine (not distroless/scratch) because thumbnailing and metadata extraction
|
||||||
|
# invoke external processes (vipsthumbnail, ffmpeg, exiftool) that must be present
|
||||||
|
# on the runtime image.
|
||||||
|
# -----------------------------------------------------------------------------
|
||||||
|
FROM alpine:3.21 AS runtime
|
||||||
|
|
||||||
|
# vips-tools: fast, low-memory image thumbnails (shrink-on-load, so multi-hundred-
|
||||||
|
# Mpx photos cost little). ffmpeg: video frame extraction. exiftool: rich metadata.
|
||||||
|
# ca-certificates/tzdata: TLS + time zones.
|
||||||
|
RUN apk add --no-cache vips-tools ffmpeg exiftool ca-certificates tzdata
|
||||||
|
|
||||||
|
# Run as an unprivileged user.
|
||||||
|
RUN addgroup -S -g 42776 tanabata && adduser -S -G tanabata -u 42776 tanabata
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
|
||||||
|
# The built SPA, served by the Go binary (matches STATIC_DIR below).
|
||||||
|
COPY --from=frontend --chown=tanabata:tanabata /src/frontend/build /app/static
|
||||||
|
# The server binary.
|
||||||
|
COPY --from=backend --chown=tanabata:tanabata /out/server /app/server
|
||||||
|
# The dedup maintenance CLI (run via `docker exec`, not the entrypoint).
|
||||||
|
COPY --from=backend --chown=tanabata:tanabata /out/dedup /app/dedup
|
||||||
|
|
||||||
|
# Data directories (overridable via FILES_PATH/THUMBS_CACHE_PATH/IMPORT_PATH).
|
||||||
|
# Created and owned by the tanabata user so a fresh named volume inherits write access.
|
||||||
|
RUN mkdir -p /data/files /data/thumbs /data/import && chown -R tanabata:tanabata /data
|
||||||
|
|
||||||
|
# Non-secret defaults mirroring .env.example. Secrets (JWT_SECRET, ADMIN_PASSWORD,
|
||||||
|
# DATABASE_URL) are intentionally NOT baked in — pass them at `docker run`.
|
||||||
|
ENV LISTEN_ADDR=:42776 \
|
||||||
|
STATIC_DIR=/app/static \
|
||||||
|
FILES_PATH=/data/files \
|
||||||
|
THUMBS_CACHE_PATH=/data/thumbs \
|
||||||
|
IMPORT_PATH=/data/import
|
||||||
|
|
||||||
|
EXPOSE 42776
|
||||||
|
VOLUME ["/data"]
|
||||||
|
USER tanabata
|
||||||
|
|
||||||
|
HEALTHCHECK --interval=30s --timeout=3s --start-period=10s --retries=3 \
|
||||||
|
CMD wget -qO- http://127.0.0.1:42776/health >/dev/null 2>&1 || exit 1
|
||||||
|
|
||||||
|
ENTRYPOINT ["/app/server"]
|
||||||
@@ -0,0 +1,661 @@
|
|||||||
|
GNU AFFERO GENERAL PUBLIC LICENSE
|
||||||
|
Version 3, 19 November 2007
|
||||||
|
|
||||||
|
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
|
||||||
|
Everyone is permitted to copy and distribute verbatim copies
|
||||||
|
of this license document, but changing it is not allowed.
|
||||||
|
|
||||||
|
Preamble
|
||||||
|
|
||||||
|
The GNU Affero General Public License is a free, copyleft license for
|
||||||
|
software and other kinds of works, specifically designed to ensure
|
||||||
|
cooperation with the community in the case of network server software.
|
||||||
|
|
||||||
|
The licenses for most software and other practical works are designed
|
||||||
|
to take away your freedom to share and change the works. By contrast,
|
||||||
|
our General Public Licenses are intended to guarantee your freedom to
|
||||||
|
share and change all versions of a program--to make sure it remains free
|
||||||
|
software for all its users.
|
||||||
|
|
||||||
|
When we speak of free software, we are referring to freedom, not
|
||||||
|
price. Our General Public Licenses are designed to make sure that you
|
||||||
|
have the freedom to distribute copies of free software (and charge for
|
||||||
|
them if you wish), that you receive source code or can get it if you
|
||||||
|
want it, that you can change the software or use pieces of it in new
|
||||||
|
free programs, and that you know you can do these things.
|
||||||
|
|
||||||
|
Developers that use our General Public Licenses protect your rights
|
||||||
|
with two steps: (1) assert copyright on the software, and (2) offer
|
||||||
|
you this License which gives you legal permission to copy, distribute
|
||||||
|
and/or modify the software.
|
||||||
|
|
||||||
|
A secondary benefit of defending all users' freedom is that
|
||||||
|
improvements made in alternate versions of the program, if they
|
||||||
|
receive widespread use, become available for other developers to
|
||||||
|
incorporate. Many developers of free software are heartened and
|
||||||
|
encouraged by the resulting cooperation. However, in the case of
|
||||||
|
software used on network servers, this result may fail to come about.
|
||||||
|
The GNU General Public License permits making a modified version and
|
||||||
|
letting the public access it on a server without ever releasing its
|
||||||
|
source code to the public.
|
||||||
|
|
||||||
|
The GNU Affero General Public License is designed specifically to
|
||||||
|
ensure that, in such cases, the modified source code becomes available
|
||||||
|
to the community. It requires the operator of a network server to
|
||||||
|
provide the source code of the modified version running there to the
|
||||||
|
users of that server. Therefore, public use of a modified version, on
|
||||||
|
a publicly accessible server, gives the public access to the source
|
||||||
|
code of the modified version.
|
||||||
|
|
||||||
|
An older license, called the Affero General Public License and
|
||||||
|
published by Affero, was designed to accomplish similar goals. This is
|
||||||
|
a different license, not a version of the Affero GPL, but Affero has
|
||||||
|
released a new version of the Affero GPL which permits relicensing under
|
||||||
|
this license.
|
||||||
|
|
||||||
|
The precise terms and conditions for copying, distribution and
|
||||||
|
modification follow.
|
||||||
|
|
||||||
|
TERMS AND CONDITIONS
|
||||||
|
|
||||||
|
0. Definitions.
|
||||||
|
|
||||||
|
"This License" refers to version 3 of the GNU Affero General Public License.
|
||||||
|
|
||||||
|
"Copyright" also means copyright-like laws that apply to other kinds of
|
||||||
|
works, such as semiconductor masks.
|
||||||
|
|
||||||
|
"The Program" refers to any copyrightable work licensed under this
|
||||||
|
License. Each licensee is addressed as "you". "Licensees" and
|
||||||
|
"recipients" may be individuals or organizations.
|
||||||
|
|
||||||
|
To "modify" a work means to copy from or adapt all or part of the work
|
||||||
|
in a fashion requiring copyright permission, other than the making of an
|
||||||
|
exact copy. The resulting work is called a "modified version" of the
|
||||||
|
earlier work or a work "based on" the earlier work.
|
||||||
|
|
||||||
|
A "covered work" means either the unmodified Program or a work based
|
||||||
|
on the Program.
|
||||||
|
|
||||||
|
To "propagate" a work means to do anything with it that, without
|
||||||
|
permission, would make you directly or secondarily liable for
|
||||||
|
infringement under applicable copyright law, except executing it on a
|
||||||
|
computer or modifying a private copy. Propagation includes copying,
|
||||||
|
distribution (with or without modification), making available to the
|
||||||
|
public, and in some countries other activities as well.
|
||||||
|
|
||||||
|
To "convey" a work means any kind of propagation that enables other
|
||||||
|
parties to make or receive copies. Mere interaction with a user through
|
||||||
|
a computer network, with no transfer of a copy, is not conveying.
|
||||||
|
|
||||||
|
An interactive user interface displays "Appropriate Legal Notices"
|
||||||
|
to the extent that it includes a convenient and prominently visible
|
||||||
|
feature that (1) displays an appropriate copyright notice, and (2)
|
||||||
|
tells the user that there is no warranty for the work (except to the
|
||||||
|
extent that warranties are provided), that licensees may convey the
|
||||||
|
work under this License, and how to view a copy of this License. If
|
||||||
|
the interface presents a list of user commands or options, such as a
|
||||||
|
menu, a prominent item in the list meets this criterion.
|
||||||
|
|
||||||
|
1. Source Code.
|
||||||
|
|
||||||
|
The "source code" for a work means the preferred form of the work
|
||||||
|
for making modifications to it. "Object code" means any non-source
|
||||||
|
form of a work.
|
||||||
|
|
||||||
|
A "Standard Interface" means an interface that either is an official
|
||||||
|
standard defined by a recognized standards body, or, in the case of
|
||||||
|
interfaces specified for a particular programming language, one that
|
||||||
|
is widely used among developers working in that language.
|
||||||
|
|
||||||
|
The "System Libraries" of an executable work include anything, other
|
||||||
|
than the work as a whole, that (a) is included in the normal form of
|
||||||
|
packaging a Major Component, but which is not part of that Major
|
||||||
|
Component, and (b) serves only to enable use of the work with that
|
||||||
|
Major Component, or to implement a Standard Interface for which an
|
||||||
|
implementation is available to the public in source code form. A
|
||||||
|
"Major Component", in this context, means a major essential component
|
||||||
|
(kernel, window system, and so on) of the specific operating system
|
||||||
|
(if any) on which the executable work runs, or a compiler used to
|
||||||
|
produce the work, or an object code interpreter used to run it.
|
||||||
|
|
||||||
|
The "Corresponding Source" for a work in object code form means all
|
||||||
|
the source code needed to generate, install, and (for an executable
|
||||||
|
work) run the object code and to modify the work, including scripts to
|
||||||
|
control those activities. However, it does not include the work's
|
||||||
|
System Libraries, or general-purpose tools or generally available free
|
||||||
|
programs which are used unmodified in performing those activities but
|
||||||
|
which are not part of the work. For example, Corresponding Source
|
||||||
|
includes interface definition files associated with source files for
|
||||||
|
the work, and the source code for shared libraries and dynamically
|
||||||
|
linked subprograms that the work is specifically designed to require,
|
||||||
|
such as by intimate data communication or control flow between those
|
||||||
|
subprograms and other parts of the work.
|
||||||
|
|
||||||
|
The Corresponding Source need not include anything that users
|
||||||
|
can regenerate automatically from other parts of the Corresponding
|
||||||
|
Source.
|
||||||
|
|
||||||
|
The Corresponding Source for a work in source code form is that
|
||||||
|
same work.
|
||||||
|
|
||||||
|
2. Basic Permissions.
|
||||||
|
|
||||||
|
All rights granted under this License are granted for the term of
|
||||||
|
copyright on the Program, and are irrevocable provided the stated
|
||||||
|
conditions are met. This License explicitly affirms your unlimited
|
||||||
|
permission to run the unmodified Program. The output from running a
|
||||||
|
covered work is covered by this License only if the output, given its
|
||||||
|
content, constitutes a covered work. This License acknowledges your
|
||||||
|
rights of fair use or other equivalent, as provided by copyright law.
|
||||||
|
|
||||||
|
You may make, run and propagate covered works that you do not
|
||||||
|
convey, without conditions so long as your license otherwise remains
|
||||||
|
in force. You may convey covered works to others for the sole purpose
|
||||||
|
of having them make modifications exclusively for you, or provide you
|
||||||
|
with facilities for running those works, provided that you comply with
|
||||||
|
the terms of this License in conveying all material for which you do
|
||||||
|
not control copyright. Those thus making or running the covered works
|
||||||
|
for you must do so exclusively on your behalf, under your direction
|
||||||
|
and control, on terms that prohibit them from making any copies of
|
||||||
|
your copyrighted material outside their relationship with you.
|
||||||
|
|
||||||
|
Conveying under any other circumstances is permitted solely under
|
||||||
|
the conditions stated below. Sublicensing is not allowed; section 10
|
||||||
|
makes it unnecessary.
|
||||||
|
|
||||||
|
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
|
||||||
|
|
||||||
|
No covered work shall be deemed part of an effective technological
|
||||||
|
measure under any applicable law fulfilling obligations under article
|
||||||
|
11 of the WIPO copyright treaty adopted on 20 December 1996, or
|
||||||
|
similar laws prohibiting or restricting circumvention of such
|
||||||
|
measures.
|
||||||
|
|
||||||
|
When you convey a covered work, you waive any legal power to forbid
|
||||||
|
circumvention of technological measures to the extent such circumvention
|
||||||
|
is effected by exercising rights under this License with respect to
|
||||||
|
the covered work, and you disclaim any intention to limit operation or
|
||||||
|
modification of the work as a means of enforcing, against the work's
|
||||||
|
users, your or third parties' legal rights to forbid circumvention of
|
||||||
|
technological measures.
|
||||||
|
|
||||||
|
4. Conveying Verbatim Copies.
|
||||||
|
|
||||||
|
You may convey verbatim copies of the Program's source code as you
|
||||||
|
receive it, in any medium, provided that you conspicuously and
|
||||||
|
appropriately publish on each copy an appropriate copyright notice;
|
||||||
|
keep intact all notices stating that this License and any
|
||||||
|
non-permissive terms added in accord with section 7 apply to the code;
|
||||||
|
keep intact all notices of the absence of any warranty; and give all
|
||||||
|
recipients a copy of this License along with the Program.
|
||||||
|
|
||||||
|
You may charge any price or no price for each copy that you convey,
|
||||||
|
and you may offer support or warranty protection for a fee.
|
||||||
|
|
||||||
|
5. Conveying Modified Source Versions.
|
||||||
|
|
||||||
|
You may convey a work based on the Program, or the modifications to
|
||||||
|
produce it from the Program, in the form of source code under the
|
||||||
|
terms of section 4, provided that you also meet all of these conditions:
|
||||||
|
|
||||||
|
a) The work must carry prominent notices stating that you modified
|
||||||
|
it, and giving a relevant date.
|
||||||
|
|
||||||
|
b) The work must carry prominent notices stating that it is
|
||||||
|
released under this License and any conditions added under section
|
||||||
|
7. This requirement modifies the requirement in section 4 to
|
||||||
|
"keep intact all notices".
|
||||||
|
|
||||||
|
c) You must license the entire work, as a whole, under this
|
||||||
|
License to anyone who comes into possession of a copy. This
|
||||||
|
License will therefore apply, along with any applicable section 7
|
||||||
|
additional terms, to the whole of the work, and all its parts,
|
||||||
|
regardless of how they are packaged. This License gives no
|
||||||
|
permission to license the work in any other way, but it does not
|
||||||
|
invalidate such permission if you have separately received it.
|
||||||
|
|
||||||
|
d) If the work has interactive user interfaces, each must display
|
||||||
|
Appropriate Legal Notices; however, if the Program has interactive
|
||||||
|
interfaces that do not display Appropriate Legal Notices, your
|
||||||
|
work need not make them do so.
|
||||||
|
|
||||||
|
A compilation of a covered work with other separate and independent
|
||||||
|
works, which are not by their nature extensions of the covered work,
|
||||||
|
and which are not combined with it such as to form a larger program,
|
||||||
|
in or on a volume of a storage or distribution medium, is called an
|
||||||
|
"aggregate" if the compilation and its resulting copyright are not
|
||||||
|
used to limit the access or legal rights of the compilation's users
|
||||||
|
beyond what the individual works permit. Inclusion of a covered work
|
||||||
|
in an aggregate does not cause this License to apply to the other
|
||||||
|
parts of the aggregate.
|
||||||
|
|
||||||
|
6. Conveying Non-Source Forms.
|
||||||
|
|
||||||
|
You may convey a covered work in object code form under the terms
|
||||||
|
of sections 4 and 5, provided that you also convey the
|
||||||
|
machine-readable Corresponding Source under the terms of this License,
|
||||||
|
in one of these ways:
|
||||||
|
|
||||||
|
a) Convey the object code in, or embodied in, a physical product
|
||||||
|
(including a physical distribution medium), accompanied by the
|
||||||
|
Corresponding Source fixed on a durable physical medium
|
||||||
|
customarily used for software interchange.
|
||||||
|
|
||||||
|
b) Convey the object code in, or embodied in, a physical product
|
||||||
|
(including a physical distribution medium), accompanied by a
|
||||||
|
written offer, valid for at least three years and valid for as
|
||||||
|
long as you offer spare parts or customer support for that product
|
||||||
|
model, to give anyone who possesses the object code either (1) a
|
||||||
|
copy of the Corresponding Source for all the software in the
|
||||||
|
product that is covered by this License, on a durable physical
|
||||||
|
medium customarily used for software interchange, for a price no
|
||||||
|
more than your reasonable cost of physically performing this
|
||||||
|
conveying of source, or (2) access to copy the
|
||||||
|
Corresponding Source from a network server at no charge.
|
||||||
|
|
||||||
|
c) Convey individual copies of the object code with a copy of the
|
||||||
|
written offer to provide the Corresponding Source. This
|
||||||
|
alternative is allowed only occasionally and noncommercially, and
|
||||||
|
only if you received the object code with such an offer, in accord
|
||||||
|
with subsection 6b.
|
||||||
|
|
||||||
|
d) Convey the object code by offering access from a designated
|
||||||
|
place (gratis or for a charge), and offer equivalent access to the
|
||||||
|
Corresponding Source in the same way through the same place at no
|
||||||
|
further charge. You need not require recipients to copy the
|
||||||
|
Corresponding Source along with the object code. If the place to
|
||||||
|
copy the object code is a network server, the Corresponding Source
|
||||||
|
may be on a different server (operated by you or a third party)
|
||||||
|
that supports equivalent copying facilities, provided you maintain
|
||||||
|
clear directions next to the object code saying where to find the
|
||||||
|
Corresponding Source. Regardless of what server hosts the
|
||||||
|
Corresponding Source, you remain obligated to ensure that it is
|
||||||
|
available for as long as needed to satisfy these requirements.
|
||||||
|
|
||||||
|
e) Convey the object code using peer-to-peer transmission, provided
|
||||||
|
you inform other peers where the object code and Corresponding
|
||||||
|
Source of the work are being offered to the general public at no
|
||||||
|
charge under subsection 6d.
|
||||||
|
|
||||||
|
A separable portion of the object code, whose source code is excluded
|
||||||
|
from the Corresponding Source as a System Library, need not be
|
||||||
|
included in conveying the object code work.
|
||||||
|
|
||||||
|
A "User Product" is either (1) a "consumer product", which means any
|
||||||
|
tangible personal property which is normally used for personal, family,
|
||||||
|
or household purposes, or (2) anything designed or sold for incorporation
|
||||||
|
into a dwelling. In determining whether a product is a consumer product,
|
||||||
|
doubtful cases shall be resolved in favor of coverage. For a particular
|
||||||
|
product received by a particular user, "normally used" refers to a
|
||||||
|
typical or common use of that class of product, regardless of the status
|
||||||
|
of the particular user or of the way in which the particular user
|
||||||
|
actually uses, or expects or is expected to use, the product. A product
|
||||||
|
is a consumer product regardless of whether the product has substantial
|
||||||
|
commercial, industrial or non-consumer uses, unless such uses represent
|
||||||
|
the only significant mode of use of the product.
|
||||||
|
|
||||||
|
"Installation Information" for a User Product means any methods,
|
||||||
|
procedures, authorization keys, or other information required to install
|
||||||
|
and execute modified versions of a covered work in that User Product from
|
||||||
|
a modified version of its Corresponding Source. The information must
|
||||||
|
suffice to ensure that the continued functioning of the modified object
|
||||||
|
code is in no case prevented or interfered with solely because
|
||||||
|
modification has been made.
|
||||||
|
|
||||||
|
If you convey an object code work under this section in, or with, or
|
||||||
|
specifically for use in, a User Product, and the conveying occurs as
|
||||||
|
part of a transaction in which the right of possession and use of the
|
||||||
|
User Product is transferred to the recipient in perpetuity or for a
|
||||||
|
fixed term (regardless of how the transaction is characterized), the
|
||||||
|
Corresponding Source conveyed under this section must be accompanied
|
||||||
|
by the Installation Information. But this requirement does not apply
|
||||||
|
if neither you nor any third party retains the ability to install
|
||||||
|
modified object code on the User Product (for example, the work has
|
||||||
|
been installed in ROM).
|
||||||
|
|
||||||
|
The requirement to provide Installation Information does not include a
|
||||||
|
requirement to continue to provide support service, warranty, or updates
|
||||||
|
for a work that has been modified or installed by the recipient, or for
|
||||||
|
the User Product in which it has been modified or installed. Access to a
|
||||||
|
network may be denied when the modification itself materially and
|
||||||
|
adversely affects the operation of the network or violates the rules and
|
||||||
|
protocols for communication across the network.
|
||||||
|
|
||||||
|
Corresponding Source conveyed, and Installation Information provided,
|
||||||
|
in accord with this section must be in a format that is publicly
|
||||||
|
documented (and with an implementation available to the public in
|
||||||
|
source code form), and must require no special password or key for
|
||||||
|
unpacking, reading or copying.
|
||||||
|
|
||||||
|
7. Additional Terms.
|
||||||
|
|
||||||
|
"Additional permissions" are terms that supplement the terms of this
|
||||||
|
License by making exceptions from one or more of its conditions.
|
||||||
|
Additional permissions that are applicable to the entire Program shall
|
||||||
|
be treated as though they were included in this License, to the extent
|
||||||
|
that they are valid under applicable law. If additional permissions
|
||||||
|
apply only to part of the Program, that part may be used separately
|
||||||
|
under those permissions, but the entire Program remains governed by
|
||||||
|
this License without regard to the additional permissions.
|
||||||
|
|
||||||
|
When you convey a copy of a covered work, you may at your option
|
||||||
|
remove any additional permissions from that copy, or from any part of
|
||||||
|
it. (Additional permissions may be written to require their own
|
||||||
|
removal in certain cases when you modify the work.) You may place
|
||||||
|
additional permissions on material, added by you to a covered work,
|
||||||
|
for which you have or can give appropriate copyright permission.
|
||||||
|
|
||||||
|
Notwithstanding any other provision of this License, for material you
|
||||||
|
add to a covered work, you may (if authorized by the copyright holders of
|
||||||
|
that material) supplement the terms of this License with terms:
|
||||||
|
|
||||||
|
a) Disclaiming warranty or limiting liability differently from the
|
||||||
|
terms of sections 15 and 16 of this License; or
|
||||||
|
|
||||||
|
b) Requiring preservation of specified reasonable legal notices or
|
||||||
|
author attributions in that material or in the Appropriate Legal
|
||||||
|
Notices displayed by works containing it; or
|
||||||
|
|
||||||
|
c) Prohibiting misrepresentation of the origin of that material, or
|
||||||
|
requiring that modified versions of such material be marked in
|
||||||
|
reasonable ways as different from the original version; or
|
||||||
|
|
||||||
|
d) Limiting the use for publicity purposes of names of licensors or
|
||||||
|
authors of the material; or
|
||||||
|
|
||||||
|
e) Declining to grant rights under trademark law for use of some
|
||||||
|
trade names, trademarks, or service marks; or
|
||||||
|
|
||||||
|
f) Requiring indemnification of licensors and authors of that
|
||||||
|
material by anyone who conveys the material (or modified versions of
|
||||||
|
it) with contractual assumptions of liability to the recipient, for
|
||||||
|
any liability that these contractual assumptions directly impose on
|
||||||
|
those licensors and authors.
|
||||||
|
|
||||||
|
All other non-permissive additional terms are considered "further
|
||||||
|
restrictions" within the meaning of section 10. If the Program as you
|
||||||
|
received it, or any part of it, contains a notice stating that it is
|
||||||
|
governed by this License along with a term that is a further
|
||||||
|
restriction, you may remove that term. If a license document contains
|
||||||
|
a further restriction but permits relicensing or conveying under this
|
||||||
|
License, you may add to a covered work material governed by the terms
|
||||||
|
of that license document, provided that the further restriction does
|
||||||
|
not survive such relicensing or conveying.
|
||||||
|
|
||||||
|
If you add terms to a covered work in accord with this section, you
|
||||||
|
must place, in the relevant source files, a statement of the
|
||||||
|
additional terms that apply to those files, or a notice indicating
|
||||||
|
where to find the applicable terms.
|
||||||
|
|
||||||
|
Additional terms, permissive or non-permissive, may be stated in the
|
||||||
|
form of a separately written license, or stated as exceptions;
|
||||||
|
the above requirements apply either way.
|
||||||
|
|
||||||
|
8. Termination.
|
||||||
|
|
||||||
|
You may not propagate or modify a covered work except as expressly
|
||||||
|
provided under this License. Any attempt otherwise to propagate or
|
||||||
|
modify it is void, and will automatically terminate your rights under
|
||||||
|
this License (including any patent licenses granted under the third
|
||||||
|
paragraph of section 11).
|
||||||
|
|
||||||
|
However, if you cease all violation of this License, then your
|
||||||
|
license from a particular copyright holder is reinstated (a)
|
||||||
|
provisionally, unless and until the copyright holder explicitly and
|
||||||
|
finally terminates your license, and (b) permanently, if the copyright
|
||||||
|
holder fails to notify you of the violation by some reasonable means
|
||||||
|
prior to 60 days after the cessation.
|
||||||
|
|
||||||
|
Moreover, your license from a particular copyright holder is
|
||||||
|
reinstated permanently if the copyright holder notifies you of the
|
||||||
|
violation by some reasonable means, this is the first time you have
|
||||||
|
received notice of violation of this License (for any work) from that
|
||||||
|
copyright holder, and you cure the violation prior to 30 days after
|
||||||
|
your receipt of the notice.
|
||||||
|
|
||||||
|
Termination of your rights under this section does not terminate the
|
||||||
|
licenses of parties who have received copies or rights from you under
|
||||||
|
this License. If your rights have been terminated and not permanently
|
||||||
|
reinstated, you do not qualify to receive new licenses for the same
|
||||||
|
material under section 10.
|
||||||
|
|
||||||
|
9. Acceptance Not Required for Having Copies.
|
||||||
|
|
||||||
|
You are not required to accept this License in order to receive or
|
||||||
|
run a copy of the Program. Ancillary propagation of a covered work
|
||||||
|
occurring solely as a consequence of using peer-to-peer transmission
|
||||||
|
to receive a copy likewise does not require acceptance. However,
|
||||||
|
nothing other than this License grants you permission to propagate or
|
||||||
|
modify any covered work. These actions infringe copyright if you do
|
||||||
|
not accept this License. Therefore, by modifying or propagating a
|
||||||
|
covered work, you indicate your acceptance of this License to do so.
|
||||||
|
|
||||||
|
10. Automatic Licensing of Downstream Recipients.
|
||||||
|
|
||||||
|
Each time you convey a covered work, the recipient automatically
|
||||||
|
receives a license from the original licensors, to run, modify and
|
||||||
|
propagate that work, subject to this License. You are not responsible
|
||||||
|
for enforcing compliance by third parties with this License.
|
||||||
|
|
||||||
|
An "entity transaction" is a transaction transferring control of an
|
||||||
|
organization, or substantially all assets of one, or subdividing an
|
||||||
|
organization, or merging organizations. If propagation of a covered
|
||||||
|
work results from an entity transaction, each party to that
|
||||||
|
transaction who receives a copy of the work also receives whatever
|
||||||
|
licenses to the work the party's predecessor in interest had or could
|
||||||
|
give under the previous paragraph, plus a right to possession of the
|
||||||
|
Corresponding Source of the work from the predecessor in interest, if
|
||||||
|
the predecessor has it or can get it with reasonable efforts.
|
||||||
|
|
||||||
|
You may not impose any further restrictions on the exercise of the
|
||||||
|
rights granted or affirmed under this License. For example, you may
|
||||||
|
not impose a license fee, royalty, or other charge for exercise of
|
||||||
|
rights granted under this License, and you may not initiate litigation
|
||||||
|
(including a cross-claim or counterclaim in a lawsuit) alleging that
|
||||||
|
any patent claim is infringed by making, using, selling, offering for
|
||||||
|
sale, or importing the Program or any portion of it.
|
||||||
|
|
||||||
|
11. Patents.
|
||||||
|
|
||||||
|
A "contributor" is a copyright holder who authorizes use under this
|
||||||
|
License of the Program or a work on which the Program is based. The
|
||||||
|
work thus licensed is called the contributor's "contributor version".
|
||||||
|
|
||||||
|
A contributor's "essential patent claims" are all patent claims
|
||||||
|
owned or controlled by the contributor, whether already acquired or
|
||||||
|
hereafter acquired, that would be infringed by some manner, permitted
|
||||||
|
by this License, of making, using, or selling its contributor version,
|
||||||
|
but do not include claims that would be infringed only as a
|
||||||
|
consequence of further modification of the contributor version. For
|
||||||
|
purposes of this definition, "control" includes the right to grant
|
||||||
|
patent sublicenses in a manner consistent with the requirements of
|
||||||
|
this License.
|
||||||
|
|
||||||
|
Each contributor grants you a non-exclusive, worldwide, royalty-free
|
||||||
|
patent license under the contributor's essential patent claims, to
|
||||||
|
make, use, sell, offer for sale, import and otherwise run, modify and
|
||||||
|
propagate the contents of its contributor version.
|
||||||
|
|
||||||
|
In the following three paragraphs, a "patent license" is any express
|
||||||
|
agreement or commitment, however denominated, not to enforce a patent
|
||||||
|
(such as an express permission to practice a patent or covenant not to
|
||||||
|
sue for patent infringement). To "grant" such a patent license to a
|
||||||
|
party means to make such an agreement or commitment not to enforce a
|
||||||
|
patent against the party.
|
||||||
|
|
||||||
|
If you convey a covered work, knowingly relying on a patent license,
|
||||||
|
and the Corresponding Source of the work is not available for anyone
|
||||||
|
to copy, free of charge and under the terms of this License, through a
|
||||||
|
publicly available network server or other readily accessible means,
|
||||||
|
then you must either (1) cause the Corresponding Source to be so
|
||||||
|
available, or (2) arrange to deprive yourself of the benefit of the
|
||||||
|
patent license for this particular work, or (3) arrange, in a manner
|
||||||
|
consistent with the requirements of this License, to extend the patent
|
||||||
|
license to downstream recipients. "Knowingly relying" means you have
|
||||||
|
actual knowledge that, but for the patent license, your conveying the
|
||||||
|
covered work in a country, or your recipient's use of the covered work
|
||||||
|
in a country, would infringe one or more identifiable patents in that
|
||||||
|
country that you have reason to believe are valid.
|
||||||
|
|
||||||
|
If, pursuant to or in connection with a single transaction or
|
||||||
|
arrangement, you convey, or propagate by procuring conveyance of, a
|
||||||
|
covered work, and grant a patent license to some of the parties
|
||||||
|
receiving the covered work authorizing them to use, propagate, modify
|
||||||
|
or convey a specific copy of the covered work, then the patent license
|
||||||
|
you grant is automatically extended to all recipients of the covered
|
||||||
|
work and works based on it.
|
||||||
|
|
||||||
|
A patent license is "discriminatory" if it does not include within
|
||||||
|
the scope of its coverage, prohibits the exercise of, or is
|
||||||
|
conditioned on the non-exercise of one or more of the rights that are
|
||||||
|
specifically granted under this License. You may not convey a covered
|
||||||
|
work if you are a party to an arrangement with a third party that is
|
||||||
|
in the business of distributing software, under which you make payment
|
||||||
|
to the third party based on the extent of your activity of conveying
|
||||||
|
the work, and under which the third party grants, to any of the
|
||||||
|
parties who would receive the covered work from you, a discriminatory
|
||||||
|
patent license (a) in connection with copies of the covered work
|
||||||
|
conveyed by you (or copies made from those copies), or (b) primarily
|
||||||
|
for and in connection with specific products or compilations that
|
||||||
|
contain the covered work, unless you entered into that arrangement,
|
||||||
|
or that patent license was granted, prior to 28 March 2007.
|
||||||
|
|
||||||
|
Nothing in this License shall be construed as excluding or limiting
|
||||||
|
any implied license or other defenses to infringement that may
|
||||||
|
otherwise be available to you under applicable patent law.
|
||||||
|
|
||||||
|
12. No Surrender of Others' Freedom.
|
||||||
|
|
||||||
|
If conditions are imposed on you (whether by court order, agreement or
|
||||||
|
otherwise) that contradict the conditions of this License, they do not
|
||||||
|
excuse you from the conditions of this License. If you cannot convey a
|
||||||
|
covered work so as to satisfy simultaneously your obligations under this
|
||||||
|
License and any other pertinent obligations, then as a consequence you may
|
||||||
|
not convey it at all. For example, if you agree to terms that obligate you
|
||||||
|
to collect a royalty for further conveying from those to whom you convey
|
||||||
|
the Program, the only way you could satisfy both those terms and this
|
||||||
|
License would be to refrain entirely from conveying the Program.
|
||||||
|
|
||||||
|
13. Remote Network Interaction; Use with the GNU General Public License.
|
||||||
|
|
||||||
|
Notwithstanding any other provision of this License, if you modify the
|
||||||
|
Program, your modified version must prominently offer all users
|
||||||
|
interacting with it remotely through a computer network (if your version
|
||||||
|
supports such interaction) an opportunity to receive the Corresponding
|
||||||
|
Source of your version by providing access to the Corresponding Source
|
||||||
|
from a network server at no charge, through some standard or customary
|
||||||
|
means of facilitating copying of software. This Corresponding Source
|
||||||
|
shall include the Corresponding Source for any work covered by version 3
|
||||||
|
of the GNU General Public License that is incorporated pursuant to the
|
||||||
|
following paragraph.
|
||||||
|
|
||||||
|
Notwithstanding any other provision of this License, you have
|
||||||
|
permission to link or combine any covered work with a work licensed
|
||||||
|
under version 3 of the GNU General Public License into a single
|
||||||
|
combined work, and to convey the resulting work. The terms of this
|
||||||
|
License will continue to apply to the part which is the covered work,
|
||||||
|
but the work with which it is combined will remain governed by version
|
||||||
|
3 of the GNU General Public License.
|
||||||
|
|
||||||
|
14. Revised Versions of this License.
|
||||||
|
|
||||||
|
The Free Software Foundation may publish revised and/or new versions of
|
||||||
|
the GNU Affero General Public License from time to time. Such new versions
|
||||||
|
will be similar in spirit to the present version, but may differ in detail to
|
||||||
|
address new problems or concerns.
|
||||||
|
|
||||||
|
Each version is given a distinguishing version number. If the
|
||||||
|
Program specifies that a certain numbered version of the GNU Affero General
|
||||||
|
Public License "or any later version" applies to it, you have the
|
||||||
|
option of following the terms and conditions either of that numbered
|
||||||
|
version or of any later version published by the Free Software
|
||||||
|
Foundation. If the Program does not specify a version number of the
|
||||||
|
GNU Affero General Public License, you may choose any version ever published
|
||||||
|
by the Free Software Foundation.
|
||||||
|
|
||||||
|
If the Program specifies that a proxy can decide which future
|
||||||
|
versions of the GNU Affero General Public License can be used, that proxy's
|
||||||
|
public statement of acceptance of a version permanently authorizes you
|
||||||
|
to choose that version for the Program.
|
||||||
|
|
||||||
|
Later license versions may give you additional or different
|
||||||
|
permissions. However, no additional obligations are imposed on any
|
||||||
|
author or copyright holder as a result of your choosing to follow a
|
||||||
|
later version.
|
||||||
|
|
||||||
|
15. Disclaimer of Warranty.
|
||||||
|
|
||||||
|
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
|
||||||
|
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
|
||||||
|
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
|
||||||
|
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
|
||||||
|
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
|
||||||
|
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
|
||||||
|
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
|
||||||
|
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
|
||||||
|
|
||||||
|
16. Limitation of Liability.
|
||||||
|
|
||||||
|
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
|
||||||
|
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
|
||||||
|
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
|
||||||
|
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
|
||||||
|
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
|
||||||
|
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
|
||||||
|
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
|
||||||
|
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
|
||||||
|
SUCH DAMAGES.
|
||||||
|
|
||||||
|
17. Interpretation of Sections 15 and 16.
|
||||||
|
|
||||||
|
If the disclaimer of warranty and limitation of liability provided
|
||||||
|
above cannot be given local legal effect according to their terms,
|
||||||
|
reviewing courts shall apply local law that most closely approximates
|
||||||
|
an absolute waiver of all civil liability in connection with the
|
||||||
|
Program, unless a warranty or assumption of liability accompanies a
|
||||||
|
copy of the Program in return for a fee.
|
||||||
|
|
||||||
|
END OF TERMS AND CONDITIONS
|
||||||
|
|
||||||
|
How to Apply These Terms to Your New Programs
|
||||||
|
|
||||||
|
If you develop a new program, and you want it to be of the greatest
|
||||||
|
possible use to the public, the best way to achieve this is to make it
|
||||||
|
free software which everyone can redistribute and change under these terms.
|
||||||
|
|
||||||
|
To do so, attach the following notices to the program. It is safest
|
||||||
|
to attach them to the start of each source file to most effectively
|
||||||
|
state the exclusion of warranty; and each file should have at least
|
||||||
|
the "copyright" line and a pointer to where the full notice is found.
|
||||||
|
|
||||||
|
<one line to give the program's name and a brief idea of what it does.>
|
||||||
|
Copyright (C) <year> <name of author>
|
||||||
|
|
||||||
|
This program is free software: you can redistribute it and/or modify
|
||||||
|
it under the terms of the GNU Affero General Public License as published by
|
||||||
|
the Free Software Foundation, either version 3 of the License, or
|
||||||
|
(at your option) any later version.
|
||||||
|
|
||||||
|
This program is distributed in the hope that it will be useful,
|
||||||
|
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||||
|
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||||
|
GNU Affero General Public License for more details.
|
||||||
|
|
||||||
|
You should have received a copy of the GNU Affero General Public License
|
||||||
|
along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||||
|
|
||||||
|
Also add information on how to contact you by electronic and paper mail.
|
||||||
|
|
||||||
|
If your software can interact with users remotely through a computer
|
||||||
|
network, you should also make sure that it provides a way for users to
|
||||||
|
get its source. For example, if your program is a web application, its
|
||||||
|
interface could display a "Source" link that leads users to an archive
|
||||||
|
of the code. There are many ways you could offer source, and different
|
||||||
|
solutions will be better for different programs; see section 13 for the
|
||||||
|
specific requirements.
|
||||||
|
|
||||||
|
You should also get your employer (if you work as a programmer) or school,
|
||||||
|
if any, to sign a "copyright disclaimer" for the program, if necessary.
|
||||||
|
For more information on this, and how to apply and follow the GNU AGPL, see
|
||||||
|
<https://www.gnu.org/licenses/>.
|
||||||
@@ -0,0 +1,105 @@
|
|||||||
|
# Tanabata File Manager
|
||||||
|
|
||||||
|
A multi-user, tag-based web file manager for images and video. Go + Gin backend
|
||||||
|
(Clean Architecture, pgx, goose migrations), SvelteKit SPA frontend, PostgreSQL,
|
||||||
|
JWT auth — shipped as a single Docker image that serves both the API and the
|
||||||
|
built SPA on one port.
|
||||||
|
|
||||||
|
## Documentation
|
||||||
|
|
||||||
|
- [`openapi.yaml`](openapi.yaml) — full REST API specification
|
||||||
|
- [`docs/REQUIREMENTS.md`](docs/REQUIREMENTS.md) — product requirements
|
||||||
|
- [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) — system architecture overview
|
||||||
|
- [`docs/DEPLOY.md`](docs/DEPLOY.md) — production deploy (Gitea Actions → host)
|
||||||
|
- [`docs/GO_PROJECT_STRUCTURE.md`](docs/GO_PROJECT_STRUCTURE.md) — backend architecture
|
||||||
|
- [`docs/FRONTEND_STRUCTURE.md`](docs/FRONTEND_STRUCTURE.md) — frontend architecture
|
||||||
|
- [`.env.example`](.env.example) — every configuration variable, documented
|
||||||
|
|
||||||
|
## Quick start
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cp .env.example .env # then edit the secrets (JWT_SECRET, ADMIN_PASSWORD, …)
|
||||||
|
docker compose up -d --build
|
||||||
|
```
|
||||||
|
|
||||||
|
By default this runs the app plus a bundled PostgreSQL container
|
||||||
|
(`COMPOSE_PROFILES=with-db`). To point at a Postgres already on the host, set
|
||||||
|
`COMPOSE_PROFILES=` empty and aim `DATABASE_URL` at `host.docker.internal`. See
|
||||||
|
[`.env.example`](.env.example) for the full matrix.
|
||||||
|
|
||||||
|
The app is published on **127.0.0.1** only and expects a reverse proxy in front
|
||||||
|
(see below). The default port is **42776** — the sum of the Unicode code points
|
||||||
|
of 七夕.
|
||||||
|
|
||||||
|
## Reverse proxy (nginx)
|
||||||
|
|
||||||
|
The container publishes its port on loopback (`127.0.0.1:${APP_PORT}:42776` in
|
||||||
|
[`docker-compose.yml`](docker-compose.yml)), so a reverse proxy on the host
|
||||||
|
terminates TLS and forwards to it. Three settings matter for this app:
|
||||||
|
|
||||||
|
1. **`client_max_body_size`** — uploads go up to `MAX_UPLOAD_BYTES` (500 MiB by
|
||||||
|
default). nginx caps request bodies at **1 MiB** out of the box, so without
|
||||||
|
this every large upload fails with `413`.
|
||||||
|
2. **Forwarded headers** — the app trusts `X-Forwarded-For` only from the hops in
|
||||||
|
`TRUSTED_PROXIES` (default: loopback + Docker bridge ranges) and keys its
|
||||||
|
login/refresh rate limiter on the resulting client IP. If the proxy doesn't
|
||||||
|
send the header, every request looks like it comes from the proxy and shares
|
||||||
|
one rate-limit bucket.
|
||||||
|
3. **Streaming for big media** — turning request/response buffering off lets
|
||||||
|
large uploads stream straight to the app and lets video range-seeks work
|
||||||
|
without nginx spooling whole files to disk first.
|
||||||
|
|
||||||
|
```nginx
|
||||||
|
server {
|
||||||
|
listen 443 ssl;
|
||||||
|
server_name tanabata.example.com;
|
||||||
|
|
||||||
|
# ssl_certificate / ssl_certificate_key ... (e.g. from certbot)
|
||||||
|
|
||||||
|
# Match MAX_UPLOAD_BYTES (500 MiB default); nginx defaults to 1m → 413.
|
||||||
|
client_max_body_size 512m;
|
||||||
|
|
||||||
|
location / {
|
||||||
|
proxy_pass http://127.0.0.1:42776; # APP_PORT
|
||||||
|
proxy_http_version 1.1;
|
||||||
|
|
||||||
|
proxy_set_header Host $host;
|
||||||
|
proxy_set_header X-Real-IP $remote_addr;
|
||||||
|
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||||
|
proxy_set_header X-Forwarded-Proto $scheme;
|
||||||
|
|
||||||
|
# Stream large uploads/downloads instead of buffering to disk; keeps
|
||||||
|
# video range-seek responsive. Scope these to file/preview locations
|
||||||
|
# instead if you'd rather keep buffering for small JSON responses.
|
||||||
|
proxy_request_buffering off;
|
||||||
|
proxy_buffering off;
|
||||||
|
proxy_read_timeout 300s;
|
||||||
|
proxy_send_timeout 300s;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
If you run the app **without** a proxy and want it reachable on the LAN, drop the
|
||||||
|
`127.0.0.1:` prefix from the `ports` line in
|
||||||
|
[`docker-compose.yml`](docker-compose.yml) and adjust `TRUSTED_PROXIES`
|
||||||
|
accordingly.
|
||||||
|
|
||||||
|
## Development
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Backend
|
||||||
|
cd backend
|
||||||
|
go run ./cmd/server # dev server
|
||||||
|
go test ./... # all tests
|
||||||
|
|
||||||
|
# Frontend
|
||||||
|
cd frontend
|
||||||
|
npm run dev # Vite dev server
|
||||||
|
npm run build # production build
|
||||||
|
npm run generate:types # regenerate API types from openapi.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
## License
|
||||||
|
|
||||||
|
Licensed under the GNU Affero General Public License v3.0 or later
|
||||||
|
(AGPL-3.0-or-later). See [`LICENSE`](LICENSE) for the full text.
|
||||||
@@ -0,0 +1,217 @@
|
|||||||
|
// Command dedup is the offline maintenance tool for duplicate detection. It runs
|
||||||
|
// in two phases:
|
||||||
|
//
|
||||||
|
// hashes — compute the perceptual hash of every live image/video that has none
|
||||||
|
// yet (images from their bytes, videos from a middle frame via ffmpeg).
|
||||||
|
// pairs — rebuild data.duplicate_pairs from all current hashes.
|
||||||
|
//
|
||||||
|
// Both phases run by default; pass -hashes or -pairs to run only one. It reuses
|
||||||
|
// the server's configuration (DATABASE_URL, FILES_PATH, THUMBS_CACHE_PATH, …) and
|
||||||
|
// is safe to re-run: hashing only touches files whose phash is NULL, and the
|
||||||
|
// pairs rebuild is a full replace.
|
||||||
|
//
|
||||||
|
// go run ./cmd/dedup # hashes, then pairs
|
||||||
|
// go run ./cmd/dedup -pairs # only rebuild pairs
|
||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"flag"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"os"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"github.com/google/uuid"
|
||||||
|
|
||||||
|
"tanabata/backend/internal/config"
|
||||||
|
"tanabata/backend/internal/db/postgres"
|
||||||
|
"tanabata/backend/internal/imagehash"
|
||||||
|
"tanabata/backend/internal/service"
|
||||||
|
"tanabata/backend/internal/storage"
|
||||||
|
)
|
||||||
|
|
||||||
|
func main() {
|
||||||
|
hashesOnly := flag.Bool("hashes", false, "only (re)compute missing perceptual hashes")
|
||||||
|
pairsOnly := flag.Bool("pairs", false, "only rebuild the duplicate pairs table")
|
||||||
|
flag.Parse()
|
||||||
|
|
||||||
|
// No flag, or both, means run everything.
|
||||||
|
doHashes := *hashesOnly || !*pairsOnly
|
||||||
|
doPairs := *pairsOnly || !*hashesOnly
|
||||||
|
|
||||||
|
cfg, err := config.Load()
|
||||||
|
if err != nil {
|
||||||
|
fatal("load config", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
ctx := context.Background()
|
||||||
|
pool, err := postgres.NewPool(ctx, cfg.DatabaseURL)
|
||||||
|
if err != nil {
|
||||||
|
fatal("connect to database", err)
|
||||||
|
}
|
||||||
|
defer pool.Close()
|
||||||
|
|
||||||
|
diskStorage, err := storage.NewDiskStorage(
|
||||||
|
cfg.FilesPath, cfg.ThumbsCachePath,
|
||||||
|
cfg.ThumbWidth, cfg.ThumbHeight,
|
||||||
|
cfg.PreviewWidth, cfg.PreviewHeight,
|
||||||
|
cfg.ThumbMaxPixels, cfg.ThumbConcurrency,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
fatal("init storage", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
fileRepo := postgres.NewFileRepo(pool)
|
||||||
|
pairRepo := postgres.NewDuplicatePairRepo(pool)
|
||||||
|
dismissalRepo := postgres.NewDismissalRepo(pool)
|
||||||
|
aclRepo := postgres.NewACLRepo(pool)
|
||||||
|
auditRepo := postgres.NewAuditRepo(pool)
|
||||||
|
tagRepo := postgres.NewTagRepo(pool)
|
||||||
|
categoryRepo := postgres.NewCategoryRepo(pool)
|
||||||
|
poolRepo := postgres.NewPoolRepo(pool)
|
||||||
|
transactor := postgres.NewTransactor(pool)
|
||||||
|
|
||||||
|
aclSvc := service.NewACLService(aclRepo, fileRepo, tagRepo, categoryRepo, poolRepo, transactor)
|
||||||
|
auditSvc := service.NewAuditService(auditRepo)
|
||||||
|
dupSvc := service.NewDuplicateService(
|
||||||
|
fileRepo, pairRepo, dismissalRepo, aclSvc, auditSvc, transactor, cfg.DuplicateHashThreshold,
|
||||||
|
)
|
||||||
|
|
||||||
|
if doHashes {
|
||||||
|
if err := backfillHashes(ctx, fileRepo, diskStorage); err != nil {
|
||||||
|
fatal("backfill hashes", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if doPairs {
|
||||||
|
fmt.Printf("rebuilding duplicate pairs (threshold %d)...\n", cfg.DuplicateHashThreshold)
|
||||||
|
// total is only known once Rescan has loaded the hashes, so create the bar
|
||||||
|
// lazily on the first progress callback.
|
||||||
|
var prog *progress
|
||||||
|
if err := dupSvc.Rescan(ctx, func(done, total int) {
|
||||||
|
if prog == nil {
|
||||||
|
prog = newProgress("matching", total)
|
||||||
|
}
|
||||||
|
prog.set(done)
|
||||||
|
}); err != nil {
|
||||||
|
fatal("rescan pairs", err)
|
||||||
|
}
|
||||||
|
if prog != nil {
|
||||||
|
prog.finish()
|
||||||
|
}
|
||||||
|
fmt.Println(" done")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// backfillHashes computes and stores a perceptual hash for every live image/video
|
||||||
|
// that lacks one. Failures on individual files are counted and reported, not
|
||||||
|
// fatal, so one unreadable file doesn't abort the whole run.
|
||||||
|
func backfillHashes(ctx context.Context, files *postgres.FileRepo, store *storage.DiskStorage) error {
|
||||||
|
pending, err := files.ListMissingPHash(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
total := len(pending)
|
||||||
|
fmt.Printf("hashing %d files without a perceptual hash...\n", total)
|
||||||
|
|
||||||
|
var hashed, skipped, failed int
|
||||||
|
prog := newProgress("hashing", total)
|
||||||
|
for i, f := range pending {
|
||||||
|
ph, err := hashOne(ctx, store, f.ID, f.MIMEType)
|
||||||
|
switch {
|
||||||
|
case err != nil:
|
||||||
|
failed++
|
||||||
|
fmt.Fprintf(os.Stderr, "\n %s (%s): %v\n", f.ID, f.MIMEType, err)
|
||||||
|
case ph == nil:
|
||||||
|
skipped++ // not decodable; leave phash NULL
|
||||||
|
default:
|
||||||
|
if err := files.SetPHash(ctx, f.ID, ph); err != nil {
|
||||||
|
return fmt.Errorf("set phash for %s: %w", f.ID, err)
|
||||||
|
}
|
||||||
|
hashed++
|
||||||
|
}
|
||||||
|
prog.set(i + 1)
|
||||||
|
}
|
||||||
|
prog.finish()
|
||||||
|
fmt.Printf(" hashed %d, skipped %d, failed %d\n", hashed, skipped, failed)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// hashOne returns the perceptual hash for one file, or nil when it isn't hashable
|
||||||
|
// (e.g. an image that won't decode). Images are hashed from their bytes; videos
|
||||||
|
// from a middle frame.
|
||||||
|
func hashOne(ctx context.Context, store *storage.DiskStorage, id uuid.UUID, mime string) (*int64, error) {
|
||||||
|
switch {
|
||||||
|
case strings.HasPrefix(mime, "image/"):
|
||||||
|
rc, err := store.Read(ctx, id)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
defer rc.Close()
|
||||||
|
data, err := io.ReadAll(rc)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if h, ok := imagehash.FromBytes(data); ok {
|
||||||
|
return &h, nil
|
||||||
|
}
|
||||||
|
return nil, nil
|
||||||
|
case strings.HasPrefix(mime, "video/"):
|
||||||
|
img, err := store.VideoFrameMiddle(ctx, id)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
h := imagehash.FromImage(img)
|
||||||
|
return &h, nil
|
||||||
|
default:
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// progress renders a dependency-free progress indicator. On a TTY it draws an
|
||||||
|
// in-place bar; otherwise (pipe, cron, CI) it prints a line every 10% so logs
|
||||||
|
// stay readable instead of filling with carriage returns.
|
||||||
|
type progress struct {
|
||||||
|
label string
|
||||||
|
tty bool
|
||||||
|
total int
|
||||||
|
lastDec int // last 10%-decile printed in non-TTY mode
|
||||||
|
}
|
||||||
|
|
||||||
|
func newProgress(label string, total int) *progress {
|
||||||
|
fi, _ := os.Stdout.Stat()
|
||||||
|
tty := fi != nil && fi.Mode()&os.ModeCharDevice != 0
|
||||||
|
return &progress{label: label, tty: tty, total: total, lastDec: -1}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *progress) set(done int) {
|
||||||
|
if p.total <= 0 {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
pct := done * 100 / p.total
|
||||||
|
if p.tty {
|
||||||
|
const w = 30
|
||||||
|
filled := done * w / p.total
|
||||||
|
fmt.Printf("\r %s [%s%s] %3d%% (%d/%d)",
|
||||||
|
p.label,
|
||||||
|
strings.Repeat("#", filled), strings.Repeat("-", w-filled),
|
||||||
|
pct, done, p.total)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if dec := pct / 10; dec != p.lastDec {
|
||||||
|
p.lastDec = dec
|
||||||
|
fmt.Printf(" %s %d%% (%d/%d)\n", p.label, pct, done, p.total)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// finish ends the in-place bar with a newline (TTY only).
|
||||||
|
func (p *progress) finish() {
|
||||||
|
if p.tty && p.total > 0 {
|
||||||
|
fmt.Println()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func fatal(what string, err error) {
|
||||||
|
fmt.Fprintf(os.Stderr, "dedup: %s: %v\n", what, err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
@@ -2,9 +2,12 @@ package main
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
|
"errors"
|
||||||
"log/slog"
|
"log/slog"
|
||||||
"net/http"
|
"net/http"
|
||||||
"os"
|
"os"
|
||||||
|
"os/signal"
|
||||||
|
"syscall"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/jackc/pgx/v5/stdlib"
|
"github.com/jackc/pgx/v5/stdlib"
|
||||||
@@ -52,6 +55,7 @@ func main() {
|
|||||||
cfg.ThumbsCachePath,
|
cfg.ThumbsCachePath,
|
||||||
cfg.ThumbWidth, cfg.ThumbHeight,
|
cfg.ThumbWidth, cfg.ThumbHeight,
|
||||||
cfg.PreviewWidth, cfg.PreviewHeight,
|
cfg.PreviewWidth, cfg.PreviewHeight,
|
||||||
|
cfg.ThumbMaxPixels, cfg.ThumbConcurrency,
|
||||||
)
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
slog.Error("failed to initialise storage", "err", err)
|
slog.Error("failed to initialise storage", "err", err)
|
||||||
@@ -59,17 +63,19 @@ func main() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Repositories
|
// Repositories
|
||||||
userRepo := postgres.NewUserRepo(pool)
|
userRepo := postgres.NewUserRepo(pool)
|
||||||
sessionRepo := postgres.NewSessionRepo(pool)
|
sessionRepo := postgres.NewSessionRepo(pool)
|
||||||
fileRepo := postgres.NewFileRepo(pool)
|
fileRepo := postgres.NewFileRepo(pool)
|
||||||
mimeRepo := postgres.NewMimeRepo(pool)
|
mimeRepo := postgres.NewMimeRepo(pool)
|
||||||
aclRepo := postgres.NewACLRepo(pool)
|
aclRepo := postgres.NewACLRepo(pool)
|
||||||
auditRepo := postgres.NewAuditRepo(pool)
|
auditRepo := postgres.NewAuditRepo(pool)
|
||||||
tagRepo := postgres.NewTagRepo(pool)
|
tagRepo := postgres.NewTagRepo(pool)
|
||||||
tagRuleRepo := postgres.NewTagRuleRepo(pool)
|
tagRuleRepo := postgres.NewTagRuleRepo(pool)
|
||||||
categoryRepo := postgres.NewCategoryRepo(pool)
|
categoryRepo := postgres.NewCategoryRepo(pool)
|
||||||
poolRepo := postgres.NewPoolRepo(pool)
|
poolRepo := postgres.NewPoolRepo(pool)
|
||||||
transactor := postgres.NewTransactor(pool)
|
duplicatePairRepo := postgres.NewDuplicatePairRepo(pool)
|
||||||
|
dismissalRepo := postgres.NewDismissalRepo(pool)
|
||||||
|
transactor := postgres.NewTransactor(pool)
|
||||||
|
|
||||||
// Services
|
// Services
|
||||||
authSvc := service.NewAuthService(
|
authSvc := service.NewAuthService(
|
||||||
@@ -78,13 +84,17 @@ func main() {
|
|||||||
cfg.JWTSecret,
|
cfg.JWTSecret,
|
||||||
cfg.JWTAccessTTL,
|
cfg.JWTAccessTTL,
|
||||||
cfg.JWTRefreshTTL,
|
cfg.JWTRefreshTTL,
|
||||||
|
cfg.ContentTokenTTL,
|
||||||
)
|
)
|
||||||
aclSvc := service.NewACLService(aclRepo, fileRepo, tagRepo, categoryRepo, poolRepo, transactor)
|
aclSvc := service.NewACLService(aclRepo, fileRepo, tagRepo, categoryRepo, poolRepo, transactor)
|
||||||
auditSvc := service.NewAuditService(auditRepo)
|
auditSvc := service.NewAuditService(auditRepo)
|
||||||
tagSvc := service.NewTagService(tagRepo, tagRuleRepo, aclSvc, auditSvc, transactor)
|
tagSvc := service.NewTagService(tagRepo, tagRuleRepo, aclSvc, auditSvc, transactor)
|
||||||
categorySvc := service.NewCategoryService(categoryRepo, tagRepo, aclSvc, auditSvc)
|
categorySvc := service.NewCategoryService(categoryRepo, tagRepo, aclSvc, auditSvc)
|
||||||
poolSvc := service.NewPoolService(poolRepo, aclSvc, auditSvc)
|
poolSvc := service.NewPoolService(poolRepo, aclSvc, auditSvc)
|
||||||
fileSvc := service.NewFileService(
|
duplicateSvc := service.NewDuplicateService(
|
||||||
|
fileRepo, duplicatePairRepo, dismissalRepo, aclSvc, auditSvc, transactor, cfg.DuplicateHashThreshold,
|
||||||
|
)
|
||||||
|
fileSvc := service.NewFileService(
|
||||||
fileRepo,
|
fileRepo,
|
||||||
mimeRepo,
|
mimeRepo,
|
||||||
diskStorage,
|
diskStorage,
|
||||||
@@ -103,21 +113,28 @@ func main() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Handlers
|
// Handlers
|
||||||
authMiddleware := handler.NewAuthMiddleware(authSvc)
|
authMiddleware := handler.NewAuthMiddleware(authSvc)
|
||||||
authHandler := handler.NewAuthHandler(authSvc)
|
authHandler := handler.NewAuthHandler(authSvc)
|
||||||
fileHandler := handler.NewFileHandler(fileSvc, tagSvc, cfg.MaxUploadBytes)
|
fileHandler := handler.NewFileHandler(fileSvc, tagSvc, authSvc, cfg.MaxUploadBytes)
|
||||||
tagHandler := handler.NewTagHandler(tagSvc, fileSvc)
|
duplicateHandler := handler.NewDuplicateHandler(duplicateSvc)
|
||||||
|
tagHandler := handler.NewTagHandler(tagSvc, fileSvc)
|
||||||
categoryHandler := handler.NewCategoryHandler(categorySvc)
|
categoryHandler := handler.NewCategoryHandler(categorySvc)
|
||||||
poolHandler := handler.NewPoolHandler(poolSvc)
|
poolHandler := handler.NewPoolHandler(poolSvc)
|
||||||
userHandler := handler.NewUserHandler(userSvc)
|
userHandler := handler.NewUserHandler(userSvc)
|
||||||
aclHandler := handler.NewACLHandler(aclSvc)
|
aclHandler := handler.NewACLHandler(aclSvc)
|
||||||
auditHandler := handler.NewAuditHandler(auditSvc)
|
auditHandler := handler.NewAuditHandler(auditSvc)
|
||||||
|
|
||||||
r := handler.NewRouter(
|
r, err := handler.NewRouter(
|
||||||
authMiddleware, authHandler,
|
authMiddleware, authHandler,
|
||||||
fileHandler, tagHandler, categoryHandler, poolHandler,
|
fileHandler, duplicateHandler, tagHandler, categoryHandler, poolHandler,
|
||||||
userHandler, aclHandler, auditHandler,
|
userHandler, aclHandler, auditHandler,
|
||||||
|
cfg.StaticDir,
|
||||||
|
cfg.TrustedProxies,
|
||||||
)
|
)
|
||||||
|
if err != nil {
|
||||||
|
slog.Error("building router", "err", err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
|
||||||
// ReadHeaderTimeout bounds slow-header (Slowloris) attacks; body read/write
|
// ReadHeaderTimeout bounds slow-header (Slowloris) attacks; body read/write
|
||||||
// are left unbounded so large file uploads and downloads can stream.
|
// are left unbounded so large file uploads and downloads can stream.
|
||||||
@@ -128,9 +145,35 @@ func main() {
|
|||||||
IdleTimeout: 120 * time.Second,
|
IdleTimeout: 120 * time.Second,
|
||||||
}
|
}
|
||||||
|
|
||||||
slog.Info("starting server", "addr", cfg.ListenAddr)
|
// Trigger a graceful shutdown on SIGINT/SIGTERM (the latter is what Docker
|
||||||
if err := srv.ListenAndServe(); err != nil {
|
// sends when the container is stopped or recreated on deploy).
|
||||||
slog.Error("server error", "err", err)
|
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
|
||||||
|
defer stop()
|
||||||
|
|
||||||
|
go func() {
|
||||||
|
slog.Info("starting server", "addr", cfg.ListenAddr)
|
||||||
|
// ListenAndServe returns ErrServerClosed after a graceful Shutdown; that
|
||||||
|
// is the expected exit, not a failure.
|
||||||
|
if err := srv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
|
||||||
|
slog.Error("server error", "err", err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
|
||||||
|
<-ctx.Done()
|
||||||
|
// Restore default signal handling so a second Ctrl+C / SIGTERM force-quits
|
||||||
|
// instead of waiting on the drain.
|
||||||
|
stop()
|
||||||
|
slog.Info("shutting down", "timeout", cfg.ShutdownTimeout)
|
||||||
|
|
||||||
|
// Stop accepting new connections and let in-flight requests finish, up to the
|
||||||
|
// timeout. Docker's stop grace period reads the same SHUTDOWN_TIMEOUT, so it
|
||||||
|
// won't SIGKILL before this returns.
|
||||||
|
shutdownCtx, cancel := context.WithTimeout(context.Background(), cfg.ShutdownTimeout)
|
||||||
|
defer cancel()
|
||||||
|
if err := srv.Shutdown(shutdownCtx); err != nil {
|
||||||
|
slog.Error("graceful shutdown failed", "err", err)
|
||||||
os.Exit(1)
|
os.Exit(1)
|
||||||
}
|
}
|
||||||
|
slog.Info("shutdown complete")
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -5,6 +5,7 @@ import (
|
|||||||
"fmt"
|
"fmt"
|
||||||
"os"
|
"os"
|
||||||
"strconv"
|
"strconv"
|
||||||
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/joho/godotenv"
|
"github.com/joho/godotenv"
|
||||||
@@ -17,6 +18,26 @@ type Config struct {
|
|||||||
JWTSecret string
|
JWTSecret string
|
||||||
JWTAccessTTL time.Duration
|
JWTAccessTTL time.Duration
|
||||||
JWTRefreshTTL time.Duration
|
JWTRefreshTTL time.Duration
|
||||||
|
// ContentTokenTTL is how long a content token stays valid. The token is a
|
||||||
|
// single-file capability used to open or stream an original by URL (e.g. a
|
||||||
|
// long video in a new tab); it is deliberately longer-lived than the access
|
||||||
|
// token and independent of the session, so playback survives access-token
|
||||||
|
// expiry and refresh rotation. Keep it only as long as a viewing session
|
||||||
|
// plausibly lasts — it is a bearer credential for that one file until expiry.
|
||||||
|
ContentTokenTTL time.Duration
|
||||||
|
// ShutdownTimeout bounds how long a graceful shutdown waits for in-flight
|
||||||
|
// requests to finish after a SIGINT/SIGTERM before the process exits. Keep it
|
||||||
|
// in step with the container's stop grace period — docker-compose.yml reads
|
||||||
|
// the same SHUTDOWN_TIMEOUT for `stop_grace_period`, so Docker doesn't SIGKILL
|
||||||
|
// mid-drain. A long upload/stream can still be cut if it outlasts this window.
|
||||||
|
ShutdownTimeout time.Duration
|
||||||
|
// TrustedProxies lists the reverse-proxy hops (CIDRs or IPs) whose
|
||||||
|
// X-Forwarded-For header is trusted. The auth rate limiter keys on the
|
||||||
|
// client IP, so this must match the proxy in front of the app — otherwise
|
||||||
|
// every request appears to come from the proxy (one shared bucket) or a
|
||||||
|
// direct caller could forge the header. Default covers loopback and the
|
||||||
|
// Docker bridge ranges a host reverse proxy reaches the container through.
|
||||||
|
TrustedProxies []string
|
||||||
|
|
||||||
// Initial admin bootstrap (applied on startup if the user does not exist)
|
// Initial admin bootstrap (applied on startup if the user does not exist)
|
||||||
AdminUsername string
|
AdminUsername string
|
||||||
@@ -35,9 +56,29 @@ type Config struct {
|
|||||||
ThumbHeight int
|
ThumbHeight int
|
||||||
PreviewWidth int
|
PreviewWidth int
|
||||||
PreviewHeight int
|
PreviewHeight int
|
||||||
|
// ThumbMaxPixels caps the pixel count of a source image decoded in-process by
|
||||||
|
// the pure-Go fallback (a decompression-bomb guard and memory bound); larger
|
||||||
|
// images then get a placeholder. It does not apply when vipsthumbnail is
|
||||||
|
// installed, which shrinks on load regardless of source size.
|
||||||
|
ThumbMaxPixels int
|
||||||
|
// ThumbConcurrency bounds how many thumbnails/previews are generated at once,
|
||||||
|
// so a burst of large images can't saturate every core or exhaust RAM. 0 =
|
||||||
|
// auto (half the available CPUs).
|
||||||
|
ThumbConcurrency int
|
||||||
|
|
||||||
// Import
|
// Import
|
||||||
ImportPath string
|
ImportPath string
|
||||||
|
|
||||||
|
// DuplicateHashThreshold is the maximum Hamming distance (out of 64) between
|
||||||
|
// two perceptual hashes for the files to be treated as duplicate candidates.
|
||||||
|
// Lower = stricter (fewer, more confident matches); higher = looser. Used only
|
||||||
|
// by the dedup rescan that (re)builds data.duplicate_pairs.
|
||||||
|
DuplicateHashThreshold int
|
||||||
|
|
||||||
|
// Static SPA. When set, the server serves the built frontend (and falls
|
||||||
|
// back to index.html for client routes) on the same port as the API. Empty
|
||||||
|
// in local development, where the Vite dev server serves the UI separately.
|
||||||
|
StaticDir string
|
||||||
}
|
}
|
||||||
|
|
||||||
// Load reads a .env file (if present) then loads all configuration from
|
// Load reads a .env file (if present) then loads all configuration from
|
||||||
@@ -63,6 +104,10 @@ func Load() (*Config, error) {
|
|||||||
return def
|
return def
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// parseDuration parses a duration env var. Every duration in this config is a
|
||||||
|
// token TTL, which must be strictly positive — a zero/negative TTL would mint
|
||||||
|
// already-expired tokens (no login, no media playback) — so reject those here
|
||||||
|
// rather than fail mysteriously at runtime.
|
||||||
parseDuration := func(key, def string) time.Duration {
|
parseDuration := func(key, def string) time.Duration {
|
||||||
raw := defaultStr(key, def)
|
raw := defaultStr(key, def)
|
||||||
d, err := time.ParseDuration(raw)
|
d, err := time.ParseDuration(raw)
|
||||||
@@ -70,6 +115,10 @@ func Load() (*Config, error) {
|
|||||||
errs = append(errs, fmt.Errorf("%s: invalid duration %q: %w", key, raw, err))
|
errs = append(errs, fmt.Errorf("%s: invalid duration %q: %w", key, raw, err))
|
||||||
return 0
|
return 0
|
||||||
}
|
}
|
||||||
|
if d <= 0 {
|
||||||
|
errs = append(errs, fmt.Errorf("%s must be positive, got %q", key, raw))
|
||||||
|
return 0
|
||||||
|
}
|
||||||
return d
|
return d
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -86,6 +135,18 @@ func Load() (*Config, error) {
|
|||||||
return n
|
return n
|
||||||
}
|
}
|
||||||
|
|
||||||
|
parseCSV := func(key, def string) []string {
|
||||||
|
raw := defaultStr(key, def)
|
||||||
|
parts := strings.Split(raw, ",")
|
||||||
|
out := make([]string, 0, len(parts))
|
||||||
|
for _, p := range parts {
|
||||||
|
if p = strings.TrimSpace(p); p != "" {
|
||||||
|
out = append(out, p)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
parseInt64 := func(key string, def int64) int64 {
|
parseInt64 := func(key string, def int64) int64 {
|
||||||
raw := os.Getenv(key)
|
raw := os.Getenv(key)
|
||||||
if raw == "" {
|
if raw == "" {
|
||||||
@@ -100,11 +161,17 @@ func Load() (*Config, error) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
cfg := &Config{
|
cfg := &Config{
|
||||||
ListenAddr: defaultStr("LISTEN_ADDR", ":8080"),
|
ListenAddr: defaultStr("LISTEN_ADDR", ":42776"),
|
||||||
JWTSecret: requireStr("JWT_SECRET"),
|
JWTSecret: requireStr("JWT_SECRET"),
|
||||||
JWTAccessTTL: parseDuration("JWT_ACCESS_TTL", "15m"),
|
JWTAccessTTL: parseDuration("JWT_ACCESS_TTL", "15m"),
|
||||||
JWTRefreshTTL: parseDuration("JWT_REFRESH_TTL", "720h"),
|
JWTRefreshTTL: parseDuration("JWT_REFRESH_TTL", "720h"),
|
||||||
|
|
||||||
|
ContentTokenTTL: parseDuration("CONTENT_TOKEN_TTL", "6h"),
|
||||||
|
|
||||||
|
ShutdownTimeout: parseDuration("SHUTDOWN_TIMEOUT", "15s"),
|
||||||
|
|
||||||
|
TrustedProxies: parseCSV("TRUSTED_PROXIES", "127.0.0.1/32,::1/128,172.16.0.0/12"),
|
||||||
|
|
||||||
AdminUsername: defaultStr("ADMIN_USERNAME", "admin"),
|
AdminUsername: defaultStr("ADMIN_USERNAME", "admin"),
|
||||||
AdminPassword: requireStr("ADMIN_PASSWORD"),
|
AdminPassword: requireStr("ADMIN_PASSWORD"),
|
||||||
|
|
||||||
@@ -114,12 +181,18 @@ func Load() (*Config, error) {
|
|||||||
ThumbsCachePath: requireStr("THUMBS_CACHE_PATH"),
|
ThumbsCachePath: requireStr("THUMBS_CACHE_PATH"),
|
||||||
MaxUploadBytes: parseInt64("MAX_UPLOAD_BYTES", 500<<20), // 500 MiB
|
MaxUploadBytes: parseInt64("MAX_UPLOAD_BYTES", 500<<20), // 500 MiB
|
||||||
|
|
||||||
ThumbWidth: parseInt("THUMB_WIDTH", 160),
|
ThumbWidth: parseInt("THUMB_WIDTH", 160),
|
||||||
ThumbHeight: parseInt("THUMB_HEIGHT", 160),
|
ThumbHeight: parseInt("THUMB_HEIGHT", 160),
|
||||||
PreviewWidth: parseInt("PREVIEW_WIDTH", 1920),
|
PreviewWidth: parseInt("PREVIEW_WIDTH", 1920),
|
||||||
PreviewHeight: parseInt("PREVIEW_HEIGHT", 1080),
|
PreviewHeight: parseInt("PREVIEW_HEIGHT", 1080),
|
||||||
|
ThumbMaxPixels: parseInt("THUMB_MAX_PIXELS", 300_000_000), // ~300 Mpx (e.g. 13000×17000)
|
||||||
|
ThumbConcurrency: parseInt("THUMB_CONCURRENCY", 0), // 0 = auto
|
||||||
|
|
||||||
ImportPath: requireStr("IMPORT_PATH"),
|
ImportPath: requireStr("IMPORT_PATH"),
|
||||||
|
|
||||||
|
DuplicateHashThreshold: parseInt("DUPLICATE_HASH_THRESHOLD", 10),
|
||||||
|
|
||||||
|
StaticDir: defaultStr("STATIC_DIR", ""),
|
||||||
}
|
}
|
||||||
|
|
||||||
if len(errs) > 0 {
|
if len(errs) > 0 {
|
||||||
|
|||||||
@@ -0,0 +1,57 @@
|
|||||||
|
package config
|
||||||
|
|
||||||
|
import (
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
// setValidEnv sets every required variable to a valid dummy value, so a test can
|
||||||
|
// then override one var to exercise a single validation path.
|
||||||
|
func setValidEnv(t *testing.T) {
|
||||||
|
t.Helper()
|
||||||
|
t.Setenv("JWT_SECRET", "test-secret")
|
||||||
|
t.Setenv("ADMIN_PASSWORD", "test-password")
|
||||||
|
t.Setenv("DATABASE_URL", "postgres://u:p@localhost:5432/db?sslmode=disable")
|
||||||
|
t.Setenv("FILES_PATH", "/tmp/files")
|
||||||
|
t.Setenv("THUMBS_CACHE_PATH", "/tmp/thumbs")
|
||||||
|
t.Setenv("IMPORT_PATH", "/tmp/import")
|
||||||
|
// Pin the TTLs to valid values so an ambient env var can't perturb the case
|
||||||
|
// under test; individual tests override the one they exercise.
|
||||||
|
t.Setenv("JWT_ACCESS_TTL", "15m")
|
||||||
|
t.Setenv("JWT_REFRESH_TTL", "720h")
|
||||||
|
t.Setenv("CONTENT_TOKEN_TTL", "6h")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLoadValid(t *testing.T) {
|
||||||
|
setValidEnv(t)
|
||||||
|
cfg, err := Load()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Load: %v", err)
|
||||||
|
}
|
||||||
|
if cfg.JWTAccessTTL <= 0 || cfg.JWTRefreshTTL <= 0 || cfg.ContentTokenTTL <= 0 {
|
||||||
|
t.Fatalf("TTLs should be positive: access=%v refresh=%v content=%v",
|
||||||
|
cfg.JWTAccessTTL, cfg.JWTRefreshTTL, cfg.ContentTokenTTL)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLoadRejectsNonPositiveTTL(t *testing.T) {
|
||||||
|
cases := []struct{ key, val string }{
|
||||||
|
{"JWT_ACCESS_TTL", "0"},
|
||||||
|
{"JWT_REFRESH_TTL", "-1h"},
|
||||||
|
{"CONTENT_TOKEN_TTL", "0s"},
|
||||||
|
}
|
||||||
|
for _, tc := range cases {
|
||||||
|
t.Run(tc.key, func(t *testing.T) {
|
||||||
|
setValidEnv(t)
|
||||||
|
t.Setenv(tc.key, tc.val)
|
||||||
|
|
||||||
|
_, err := Load()
|
||||||
|
if err == nil {
|
||||||
|
t.Fatalf("expected error for %s=%q", tc.key, tc.val)
|
||||||
|
}
|
||||||
|
if !strings.Contains(err.Error(), tc.key) || !strings.Contains(err.Error(), "must be positive") {
|
||||||
|
t.Fatalf("error should name %s and mention positivity, got: %v", tc.key, err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -116,6 +116,12 @@ func (r *CategoryRepo) List(ctx context.Context, params port.OffsetParams) (*dom
|
|||||||
args = append(args, "%"+params.Search+"%")
|
args = append(args, "%"+params.Search+"%")
|
||||||
n++
|
n++
|
||||||
}
|
}
|
||||||
|
// Restrict to categories the viewer may see (private-by-default), unless admin.
|
||||||
|
if !params.ViewerIsAdmin {
|
||||||
|
var aclCond string
|
||||||
|
aclCond, n, args = aclVisibilityCond("c", objTypeCategory, params.ViewerID, n, args)
|
||||||
|
conditions = append(conditions, aclCond)
|
||||||
|
}
|
||||||
|
|
||||||
where := ""
|
where := ""
|
||||||
if len(conditions) > 0 {
|
if len(conditions) > 0 {
|
||||||
@@ -294,4 +300,4 @@ func (r *CategoryRepo) Delete(ctx context.Context, id uuid.UUID) error {
|
|||||||
return domain.ErrNotFound
|
return domain.ErrNotFound
|
||||||
}
|
}
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,145 @@
|
|||||||
|
package postgres
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
|
||||||
|
"github.com/google/uuid"
|
||||||
|
"github.com/jackc/pgx/v5"
|
||||||
|
"github.com/jackc/pgx/v5/pgxpool"
|
||||||
|
|
||||||
|
"tanabata/backend/internal/domain"
|
||||||
|
"tanabata/backend/internal/port"
|
||||||
|
)
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// DuplicatePairRepo
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
// DuplicatePairRepo implements port.DuplicatePairRepo using PostgreSQL.
|
||||||
|
type DuplicatePairRepo struct {
|
||||||
|
pool *pgxpool.Pool
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewDuplicatePairRepo creates a DuplicatePairRepo backed by pool.
|
||||||
|
func NewDuplicatePairRepo(pool *pgxpool.Pool) *DuplicatePairRepo {
|
||||||
|
return &DuplicatePairRepo{pool: pool}
|
||||||
|
}
|
||||||
|
|
||||||
|
var _ port.DuplicatePairRepo = (*DuplicatePairRepo)(nil)
|
||||||
|
|
||||||
|
// ReplaceAll atomically replaces the entire pairs table with the given set.
|
||||||
|
// The rescan recomputes pairs from scratch, so a full DELETE + COPY is both
|
||||||
|
// correct and the simplest way to drop pairs that no longer qualify.
|
||||||
|
func (r *DuplicatePairRepo) ReplaceAll(ctx context.Context, pairs []domain.DuplicatePair) error {
|
||||||
|
tx, err := r.pool.Begin(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("DuplicatePairRepo.ReplaceAll begin: %w", err)
|
||||||
|
}
|
||||||
|
defer tx.Rollback(ctx) //nolint:errcheck // no-op after a successful commit
|
||||||
|
|
||||||
|
if _, err := tx.Exec(ctx, `DELETE FROM data.duplicate_pairs`); err != nil {
|
||||||
|
return fmt.Errorf("DuplicatePairRepo.ReplaceAll delete: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(pairs) > 0 {
|
||||||
|
rows := make([][]any, len(pairs))
|
||||||
|
for i, p := range pairs {
|
||||||
|
rows[i] = []any{p.FileA, p.FileB, int16(p.Distance)}
|
||||||
|
}
|
||||||
|
if _, err := tx.CopyFrom(ctx,
|
||||||
|
pgx.Identifier{"data", "duplicate_pairs"},
|
||||||
|
[]string{"file_a", "file_b", "distance"},
|
||||||
|
pgx.CopyFromRows(rows),
|
||||||
|
); err != nil {
|
||||||
|
return fmt.Errorf("DuplicatePairRepo.ReplaceAll copy: %w", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := tx.Commit(ctx); err != nil {
|
||||||
|
return fmt.Errorf("DuplicatePairRepo.ReplaceAll commit: %w", err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
type pairRow struct {
|
||||||
|
FileA uuid.UUID `db:"file_a"`
|
||||||
|
FileB uuid.UUID `db:"file_b"`
|
||||||
|
Distance int16 `db:"distance"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// ListVisible returns every stored pair where both files are live (not trashed),
|
||||||
|
// the pair is not dismissed, and — for non-admins — both files are visible to the
|
||||||
|
// viewer under the private-by-default model. This is the input to clustering.
|
||||||
|
func (r *DuplicatePairRepo) ListVisible(ctx context.Context, viewerID int16, isAdmin bool) ([]domain.DuplicatePair, error) {
|
||||||
|
args := make([]any, 0, 4)
|
||||||
|
n := 1
|
||||||
|
aclWhere := ""
|
||||||
|
if !isAdmin {
|
||||||
|
var ca, cb string
|
||||||
|
ca, n, args = aclVisibilityCond("fa", objTypeFile, viewerID, n, args)
|
||||||
|
cb, n, args = aclVisibilityCond("fb", objTypeFile, viewerID, n, args)
|
||||||
|
aclWhere = "AND " + ca + " AND " + cb
|
||||||
|
}
|
||||||
|
|
||||||
|
sqlStr := fmt.Sprintf(`
|
||||||
|
SELECT p.file_a, p.file_b, p.distance
|
||||||
|
FROM data.duplicate_pairs p
|
||||||
|
JOIN data.files fa ON fa.id = p.file_a AND fa.is_deleted = false
|
||||||
|
JOIN data.files fb ON fb.id = p.file_b AND fb.is_deleted = false
|
||||||
|
WHERE NOT EXISTS (
|
||||||
|
SELECT 1 FROM data.duplicate_dismissals d
|
||||||
|
WHERE d.file_a = p.file_a AND d.file_b = p.file_b
|
||||||
|
)
|
||||||
|
%s
|
||||||
|
ORDER BY p.file_a, p.file_b`, aclWhere)
|
||||||
|
|
||||||
|
rows, err := r.pool.Query(ctx, sqlStr, args...)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("DuplicatePairRepo.ListVisible: %w", err)
|
||||||
|
}
|
||||||
|
collected, err := pgx.CollectRows(rows, pgx.RowToStructByName[pairRow])
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("DuplicatePairRepo.ListVisible scan: %w", err)
|
||||||
|
}
|
||||||
|
out := make([]domain.DuplicatePair, len(collected))
|
||||||
|
for i, row := range collected {
|
||||||
|
out[i] = domain.DuplicatePair{FileA: row.FileA, FileB: row.FileB, Distance: int(row.Distance)}
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// DismissalRepo
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
// DismissalRepo implements port.DismissalRepo using PostgreSQL.
|
||||||
|
type DismissalRepo struct {
|
||||||
|
pool *pgxpool.Pool
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewDismissalRepo creates a DismissalRepo backed by pool.
|
||||||
|
func NewDismissalRepo(pool *pgxpool.Pool) *DismissalRepo {
|
||||||
|
return &DismissalRepo{pool: pool}
|
||||||
|
}
|
||||||
|
|
||||||
|
var _ port.DismissalRepo = (*DismissalRepo)(nil)
|
||||||
|
|
||||||
|
// Add records a pair as "not a duplicate". The two ids are stored in canonical
|
||||||
|
// (file_a < file_b) order to match the table's CHECK and avoid (a,b)/(b,a)
|
||||||
|
// duplicates; a repeated dismissal is a no-op.
|
||||||
|
func (r *DismissalRepo) Add(ctx context.Context, a, b uuid.UUID, userID int16) error {
|
||||||
|
if bytes.Compare(a[:], b[:]) > 0 {
|
||||||
|
a, b = b, a
|
||||||
|
}
|
||||||
|
const sqlStr = `
|
||||||
|
INSERT INTO data.duplicate_dismissals (file_a, file_b, dismissed_by)
|
||||||
|
VALUES ($1, $2, $3)
|
||||||
|
ON CONFLICT (file_a, file_b) DO NOTHING`
|
||||||
|
q := connOrTx(ctx, r.pool)
|
||||||
|
if _, err := q.Exec(ctx, sqlStr, a, b, userID); err != nil {
|
||||||
|
return fmt.Errorf("DismissalRepo.Add: %w", err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
@@ -36,6 +36,7 @@ type fileRow struct {
|
|||||||
CreatorName string `db:"creator_name"`
|
CreatorName string `db:"creator_name"`
|
||||||
IsPublic bool `db:"is_public"`
|
IsPublic bool `db:"is_public"`
|
||||||
IsDeleted bool `db:"is_deleted"`
|
IsDeleted bool `db:"is_deleted"`
|
||||||
|
NeedsReview bool `db:"needs_review"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// fileTagRow is used for both single-file and batch tag loading.
|
// fileTagRow is used for both single-file and batch tag loading.
|
||||||
@@ -81,6 +82,7 @@ func toFile(r fileRow) domain.File {
|
|||||||
CreatorName: r.CreatorName,
|
CreatorName: r.CreatorName,
|
||||||
IsPublic: r.IsPublic,
|
IsPublic: r.IsPublic,
|
||||||
IsDeleted: r.IsDeleted,
|
IsDeleted: r.IsDeleted,
|
||||||
|
NeedsReview: r.NeedsReview,
|
||||||
CreatedAt: domain.UUIDCreatedAt(r.ID),
|
CreatedAt: domain.UUIDCreatedAt(r.ID),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -142,7 +144,7 @@ func makeCursor(r fileRow, sort, order string) fileCursor {
|
|||||||
}
|
}
|
||||||
case "mime":
|
case "mime":
|
||||||
val = r.MIMEType
|
val = r.MIMEType
|
||||||
// "created": val is empty; f.id is the sort key.
|
// "created": val is empty; f.id is the sort key.
|
||||||
}
|
}
|
||||||
return fileCursor{Sort: sort, Order: order, ID: r.ID.String(), Val: val}
|
return fileCursor{Sort: sort, Order: order, ID: r.ID.String(), Val: val}
|
||||||
}
|
}
|
||||||
@@ -293,7 +295,7 @@ const fileSelectCTE = `
|
|||||||
mt.name AS mime_type, mt.extension AS mime_extension,
|
mt.name AS mime_type, mt.extension AS mime_extension,
|
||||||
r.content_datetime, r.notes, r.metadata, r.exif, r.phash,
|
r.content_datetime, r.notes, r.metadata, r.exif, r.phash,
|
||||||
r.creator_id, u.name AS creator_name,
|
r.creator_id, u.name AS creator_name,
|
||||||
r.is_public, r.is_deleted
|
r.is_public, r.is_deleted, r.needs_review
|
||||||
FROM r
|
FROM r
|
||||||
JOIN core.mime_types mt ON mt.id = r.mime_id
|
JOIN core.mime_types mt ON mt.id = r.mime_id
|
||||||
JOIN core.users u ON u.id = r.creator_id`
|
JOIN core.users u ON u.id = r.creator_id`
|
||||||
@@ -316,7 +318,8 @@ func (r *FileRepo) Create(ctx context.Context, f *domain.File) (*domain.File, er
|
|||||||
$4, $5, $6, $7, $8, $9, $10
|
$4, $5, $6, $7, $8, $9, $10
|
||||||
)
|
)
|
||||||
RETURNING id, original_name, mime_id, content_datetime, notes,
|
RETURNING id, original_name, mime_id, content_datetime, notes,
|
||||||
metadata, exif, phash, creator_id, is_public, is_deleted
|
metadata, exif, phash, creator_id, is_public, is_deleted,
|
||||||
|
needs_review
|
||||||
)` + fileSelectCTE
|
)` + fileSelectCTE
|
||||||
|
|
||||||
q := connOrTx(ctx, r.pool)
|
q := connOrTx(ctx, r.pool)
|
||||||
@@ -346,7 +349,7 @@ func (r *FileRepo) GetByID(ctx context.Context, id uuid.UUID) (*domain.File, err
|
|||||||
mt.name AS mime_type, mt.extension AS mime_extension,
|
mt.name AS mime_type, mt.extension AS mime_extension,
|
||||||
f.content_datetime, f.notes, f.metadata, f.exif, f.phash,
|
f.content_datetime, f.notes, f.metadata, f.exif, f.phash,
|
||||||
f.creator_id, u.name AS creator_name,
|
f.creator_id, u.name AS creator_name,
|
||||||
f.is_public, f.is_deleted
|
f.is_public, f.is_deleted, f.needs_review
|
||||||
FROM data.files f
|
FROM data.files f
|
||||||
JOIN core.mime_types mt ON mt.id = f.mime_id
|
JOIN core.mime_types mt ON mt.id = f.mime_id
|
||||||
JOIN core.users u ON u.id = f.creator_id
|
JOIN core.users u ON u.id = f.creator_id
|
||||||
@@ -389,7 +392,8 @@ func (r *FileRepo) Update(ctx context.Context, id uuid.UUID, f *domain.File) (*d
|
|||||||
is_public = $6
|
is_public = $6
|
||||||
WHERE id = $1
|
WHERE id = $1
|
||||||
RETURNING id, original_name, mime_id, content_datetime, notes,
|
RETURNING id, original_name, mime_id, content_datetime, notes,
|
||||||
metadata, exif, phash, creator_id, is_public, is_deleted
|
metadata, exif, phash, creator_id, is_public, is_deleted,
|
||||||
|
needs_review
|
||||||
)` + fileSelectCTE
|
)` + fileSelectCTE
|
||||||
|
|
||||||
q := connOrTx(ctx, r.pool)
|
q := connOrTx(ctx, r.pool)
|
||||||
@@ -416,6 +420,115 @@ func (r *FileRepo) Update(ctx context.Context, id uuid.UUID, f *domain.File) (*d
|
|||||||
return &updated, nil
|
return &updated, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// SetNeedsReview sets the review status on the given files in one statement.
|
||||||
|
// Trashed files are left untouched. No-op for an empty id list.
|
||||||
|
func (r *FileRepo) SetNeedsReview(ctx context.Context, ids []uuid.UUID, value bool) error {
|
||||||
|
if len(ids) == 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
const sqlStr = `UPDATE data.files SET needs_review = $2 WHERE id = ANY($1) AND is_deleted = false`
|
||||||
|
q := connOrTx(ctx, r.pool)
|
||||||
|
if _, err := q.Exec(ctx, sqlStr, ids, value); err != nil {
|
||||||
|
return fmt.Errorf("FileRepo.SetNeedsReview: %w", err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// SetPHash sets (or clears, when phash is nil) the perceptual hash of a file.
|
||||||
|
// Used by the dedup backfill and on content replacement; phash is non-critical,
|
||||||
|
// recomputable metadata, so callers may treat failures as best-effort.
|
||||||
|
func (r *FileRepo) SetPHash(ctx context.Context, id uuid.UUID, phash *int64) error {
|
||||||
|
const sqlStr = `UPDATE data.files SET phash = $2 WHERE id = $1`
|
||||||
|
q := connOrTx(ctx, r.pool)
|
||||||
|
if _, err := q.Exec(ctx, sqlStr, id, phash); err != nil {
|
||||||
|
return fmt.Errorf("FileRepo.SetPHash: %w", err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Perceptual-hash / duplicate support
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
// ListMissingPHash returns live image/video files that have no perceptual hash
|
||||||
|
// yet — the work list for the dedup backfill. Tags are not loaded (the backfill
|
||||||
|
// only needs the id and MIME type to choose image vs video hashing).
|
||||||
|
func (r *FileRepo) ListMissingPHash(ctx context.Context) ([]domain.File, error) {
|
||||||
|
const sqlStr = `
|
||||||
|
SELECT f.id, f.original_name,
|
||||||
|
mt.name AS mime_type, mt.extension AS mime_extension,
|
||||||
|
f.content_datetime, f.notes, f.metadata, f.exif, f.phash,
|
||||||
|
f.creator_id, u.name AS creator_name,
|
||||||
|
f.is_public, f.is_deleted, f.needs_review
|
||||||
|
FROM data.files f
|
||||||
|
JOIN core.mime_types mt ON mt.id = f.mime_id
|
||||||
|
JOIN core.users u ON u.id = f.creator_id
|
||||||
|
WHERE f.phash IS NULL AND f.is_deleted = false
|
||||||
|
AND (mt.name LIKE 'image/%' OR mt.name LIKE 'video/%')
|
||||||
|
ORDER BY f.id`
|
||||||
|
|
||||||
|
q := connOrTx(ctx, r.pool)
|
||||||
|
rows, err := q.Query(ctx, sqlStr)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("FileRepo.ListMissingPHash: %w", err)
|
||||||
|
}
|
||||||
|
collected, err := pgx.CollectRows(rows, pgx.RowToStructByName[fileRow])
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("FileRepo.ListMissingPHash scan: %w", err)
|
||||||
|
}
|
||||||
|
files := make([]domain.File, len(collected))
|
||||||
|
for i, row := range collected {
|
||||||
|
files[i] = toFile(row)
|
||||||
|
}
|
||||||
|
return files, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// phashRow is the minimal projection used to build duplicate clusters.
|
||||||
|
type phashRow struct {
|
||||||
|
ID uuid.UUID `db:"id"`
|
||||||
|
PHash int64 `db:"phash"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// ListAllPHashes returns the id and perceptual hash of every live, hashed file.
|
||||||
|
// It is the global input to the dedup rescan, so it deliberately ignores ACL —
|
||||||
|
// the rescan builds the shared pairs table; visibility is enforced on read.
|
||||||
|
func (r *FileRepo) ListAllPHashes(ctx context.Context) ([]domain.PHashEntry, error) {
|
||||||
|
const sqlStr = `SELECT id, phash FROM data.files WHERE is_deleted = false AND phash IS NOT NULL`
|
||||||
|
q := connOrTx(ctx, r.pool)
|
||||||
|
rows, err := q.Query(ctx, sqlStr)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("FileRepo.ListAllPHashes: %w", err)
|
||||||
|
}
|
||||||
|
collected, err := pgx.CollectRows(rows, pgx.RowToStructByName[phashRow])
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("FileRepo.ListAllPHashes scan: %w", err)
|
||||||
|
}
|
||||||
|
out := make([]domain.PHashEntry, len(collected))
|
||||||
|
for i, row := range collected {
|
||||||
|
out[i] = domain.PHashEntry{ID: row.ID, PHash: row.PHash}
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// CopyPoolMemberships adds targetID to every pool sourceID belongs to (copying
|
||||||
|
// the source's position), skipping pools the target is already in. Used by the
|
||||||
|
// duplicate merge to preserve the discarded file's pool memberships on the
|
||||||
|
// survivor. The merge is authorised at the file level, so pool ACL is not
|
||||||
|
// re-checked here.
|
||||||
|
func (r *FileRepo) CopyPoolMemberships(ctx context.Context, targetID, sourceID uuid.UUID) error {
|
||||||
|
const sqlStr = `
|
||||||
|
INSERT INTO data.file_pool (file_id, pool_id, position)
|
||||||
|
SELECT $1, fp.pool_id, fp.position
|
||||||
|
FROM data.file_pool fp
|
||||||
|
WHERE fp.file_id = $2
|
||||||
|
ON CONFLICT (file_id, pool_id) DO NOTHING`
|
||||||
|
q := connOrTx(ctx, r.pool)
|
||||||
|
if _, err := q.Exec(ctx, sqlStr, targetID, sourceID); err != nil {
|
||||||
|
return fmt.Errorf("FileRepo.CopyPoolMemberships: %w", err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// SoftDelete / Restore / DeletePermanent
|
// SoftDelete / Restore / DeletePermanent
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
@@ -444,7 +557,8 @@ func (r *FileRepo) Restore(ctx context.Context, id uuid.UUID) (*domain.File, err
|
|||||||
SET is_deleted = false
|
SET is_deleted = false
|
||||||
WHERE id = $1 AND is_deleted = true
|
WHERE id = $1 AND is_deleted = true
|
||||||
RETURNING id, original_name, mime_id, content_datetime, notes,
|
RETURNING id, original_name, mime_id, content_datetime, notes,
|
||||||
metadata, exif, phash, creator_id, is_public, is_deleted
|
metadata, exif, phash, creator_id, is_public, is_deleted,
|
||||||
|
needs_review
|
||||||
)` + fileSelectCTE
|
)` + fileSelectCTE
|
||||||
|
|
||||||
q := connOrTx(ctx, r.pool)
|
q := connOrTx(ctx, r.pool)
|
||||||
@@ -569,7 +683,7 @@ func (r *FileRepo) List(ctx context.Context, params domain.FileListParams) (*dom
|
|||||||
cursorVal = av.OriginalName
|
cursorVal = av.OriginalName
|
||||||
case "mime":
|
case "mime":
|
||||||
cursorVal = av.MIMEType
|
cursorVal = av.MIMEType
|
||||||
// "created": cursorVal stays ""; cursorID is the sort key.
|
// "created": cursorVal stays ""; cursorID is the sort key.
|
||||||
}
|
}
|
||||||
hasCursor = true
|
hasCursor = true
|
||||||
isAnchor = true
|
isAnchor = true
|
||||||
@@ -607,6 +721,13 @@ func (r *FileRepo) List(ctx context.Context, params domain.FileListParams) (*dom
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Restrict to files the viewer may see (private-by-default), unless admin.
|
||||||
|
if !params.ViewerIsAdmin {
|
||||||
|
var aclCond string
|
||||||
|
aclCond, n, args = aclVisibilityCond("f", objTypeFile, params.ViewerID, n, args)
|
||||||
|
conds = append(conds, aclCond)
|
||||||
|
}
|
||||||
|
|
||||||
var orderBy string
|
var orderBy string
|
||||||
if hasCursor {
|
if hasCursor {
|
||||||
ksWhere, ksOrder, nextN, ksArgs := buildKeysetCond(
|
ksWhere, ksOrder, nextN, ksArgs := buildKeysetCond(
|
||||||
@@ -631,7 +752,7 @@ func (r *FileRepo) List(ctx context.Context, params domain.FileListParams) (*dom
|
|||||||
mt.name AS mime_type, mt.extension AS mime_extension,
|
mt.name AS mime_type, mt.extension AS mime_extension,
|
||||||
f.content_datetime, f.notes, f.metadata, f.exif, f.phash,
|
f.content_datetime, f.notes, f.metadata, f.exif, f.phash,
|
||||||
f.creator_id, u.name AS creator_name,
|
f.creator_id, u.name AS creator_name,
|
||||||
f.is_public, f.is_deleted
|
f.is_public, f.is_deleted, f.needs_review
|
||||||
FROM data.files f
|
FROM data.files f
|
||||||
JOIN core.mime_types mt ON mt.id = f.mime_id
|
JOIN core.mime_types mt ON mt.id = f.mime_id
|
||||||
JOIN core.users u ON u.id = f.creator_id
|
JOIN core.users u ON u.id = f.creator_id
|
||||||
@@ -794,3 +915,43 @@ func (r *FileRepo) loadTagsBatch(ctx context.Context, fileIDs []uuid.UUID) (map[
|
|||||||
}
|
}
|
||||||
return result, nil
|
return result, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// RecordView appends a row to activity.file_views. viewed_at defaults to
|
||||||
|
// statement_timestamp(), so each call records a distinct view in the history.
|
||||||
|
func (r *FileRepo) RecordView(ctx context.Context, fileID uuid.UUID, userID int16) error {
|
||||||
|
const query = `INSERT INTO activity.file_views (file_id, user_id) VALUES ($1, $2)`
|
||||||
|
if _, err := connOrTx(ctx, r.pool).Exec(ctx, query, fileID, userID); err != nil {
|
||||||
|
return fmt.Errorf("FileRepo.RecordView: %w", err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// RecordTagUses appends a row to activity.tag_uses for each tag referenced in a
|
||||||
|
// filter DSL, flagging it included (positive) or excluded (negated). Tags are
|
||||||
|
// deduplicated per call, so one statement_timestamp() never collides on the
|
||||||
|
// (tag_id, used_at, user_id) PK; ON CONFLICT DO NOTHING guards the rest. A
|
||||||
|
// filter with no tag terms is a no-op.
|
||||||
|
func (r *FileRepo) RecordTagUses(ctx context.Context, userID int16, filterDSL string) error {
|
||||||
|
uses := filterTagUses(filterDSL)
|
||||||
|
if len(uses) == 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
var sb strings.Builder
|
||||||
|
sb.WriteString("INSERT INTO activity.tag_uses (tag_id, user_id, is_included) VALUES ")
|
||||||
|
args := make([]any, 0, len(uses)*3)
|
||||||
|
for i, u := range uses {
|
||||||
|
if i > 0 {
|
||||||
|
sb.WriteString(", ")
|
||||||
|
}
|
||||||
|
base := i * 3
|
||||||
|
fmt.Fprintf(&sb, "($%d, $%d, $%d)", base+1, base+2, base+3)
|
||||||
|
args = append(args, u.tagID, userID, u.included)
|
||||||
|
}
|
||||||
|
sb.WriteString(" ON CONFLICT DO NOTHING")
|
||||||
|
|
||||||
|
if _, err := connOrTx(ctx, r.pool).Exec(ctx, sb.String(), args...); err != nil {
|
||||||
|
return fmt.Errorf("FileRepo.RecordTagUses: %w", err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|||||||
@@ -15,7 +15,7 @@ import (
|
|||||||
type filterTokenKind int
|
type filterTokenKind int
|
||||||
|
|
||||||
const (
|
const (
|
||||||
ftkAnd filterTokenKind = iota
|
ftkAnd filterTokenKind = iota
|
||||||
ftkOr
|
ftkOr
|
||||||
ftkNot
|
ftkNot
|
||||||
ftkLParen
|
ftkLParen
|
||||||
@@ -23,6 +23,7 @@ const (
|
|||||||
ftkTag // t=<uuid>
|
ftkTag // t=<uuid>
|
||||||
ftkMimeExact // m=<int>
|
ftkMimeExact // m=<int>
|
||||||
ftkMimeLike // m~<pattern>
|
ftkMimeLike // m~<pattern>
|
||||||
|
ftkReview // r=<0|1>
|
||||||
)
|
)
|
||||||
|
|
||||||
type filterToken struct {
|
type filterToken struct {
|
||||||
@@ -31,6 +32,7 @@ type filterToken struct {
|
|||||||
untagged bool // ftkTag with zero UUID → "file has no tags"
|
untagged bool // ftkTag with zero UUID → "file has no tags"
|
||||||
mimeID int16 // ftkMimeExact
|
mimeID int16 // ftkMimeExact
|
||||||
pattern string // ftkMimeLike
|
pattern string // ftkMimeLike
|
||||||
|
review bool // ftkReview → needs_review value
|
||||||
}
|
}
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
@@ -44,9 +46,9 @@ type filterNode interface {
|
|||||||
toSQL(n int, args []any) (string, int, []any)
|
toSQL(n int, args []any) (string, int, []any)
|
||||||
}
|
}
|
||||||
|
|
||||||
type andNode struct{ left, right filterNode }
|
type andNode struct{ left, right filterNode }
|
||||||
type orNode struct{ left, right filterNode }
|
type orNode struct{ left, right filterNode }
|
||||||
type notNode struct{ child filterNode }
|
type notNode struct{ child filterNode }
|
||||||
type leafNode struct{ tok filterToken }
|
type leafNode struct{ tok filterToken }
|
||||||
|
|
||||||
func (a *andNode) toSQL(n int, args []any) (string, int, []any) {
|
func (a *andNode) toSQL(n int, args []any) (string, int, []any) {
|
||||||
@@ -80,6 +82,8 @@ func (l *leafNode) toSQL(n int, args []any) (string, int, []any) {
|
|||||||
case ftkMimeLike:
|
case ftkMimeLike:
|
||||||
// mt alias comes from the JOIN in the main file query (always present).
|
// mt alias comes from the JOIN in the main file query (always present).
|
||||||
return fmt.Sprintf("mt.name LIKE $%d", n), n + 1, append(args, l.tok.pattern)
|
return fmt.Sprintf("mt.name LIKE $%d", n), n + 1, append(args, l.tok.pattern)
|
||||||
|
case ftkReview:
|
||||||
|
return fmt.Sprintf("f.needs_review = $%d", n), n + 1, append(args, l.tok.review)
|
||||||
}
|
}
|
||||||
panic("filterNode.toSQL: unknown leaf kind")
|
panic("filterNode.toSQL: unknown leaf kind")
|
||||||
}
|
}
|
||||||
@@ -130,6 +134,15 @@ func lexFilter(dsl string) ([]filterToken, error) {
|
|||||||
case strings.HasPrefix(p, "m~"):
|
case strings.HasPrefix(p, "m~"):
|
||||||
// The pattern value is passed as a query parameter, so no SQL injection risk.
|
// The pattern value is passed as a query parameter, so no SQL injection risk.
|
||||||
tokens = append(tokens, filterToken{kind: ftkMimeLike, pattern: p[2:]})
|
tokens = append(tokens, filterToken{kind: ftkMimeLike, pattern: p[2:]})
|
||||||
|
case strings.HasPrefix(p, "r="):
|
||||||
|
switch p[2:] {
|
||||||
|
case "1":
|
||||||
|
tokens = append(tokens, filterToken{kind: ftkReview, review: true})
|
||||||
|
case "0":
|
||||||
|
tokens = append(tokens, filterToken{kind: ftkReview, review: false})
|
||||||
|
default:
|
||||||
|
return nil, fmt.Errorf("filter: invalid review flag %q (want r=0 or r=1)", p[2:])
|
||||||
|
}
|
||||||
default:
|
default:
|
||||||
return nil, fmt.Errorf("filter: unknown token %q", p)
|
return nil, fmt.Errorf("filter: unknown token %q", p)
|
||||||
}
|
}
|
||||||
@@ -241,7 +254,7 @@ func (p *filterParser) parseAtom() (filterNode, error) {
|
|||||||
return expr, nil
|
return expr, nil
|
||||||
}
|
}
|
||||||
switch t.kind {
|
switch t.kind {
|
||||||
case ftkTag, ftkMimeExact, ftkMimeLike:
|
case ftkTag, ftkMimeExact, ftkMimeLike, ftkReview:
|
||||||
p.next()
|
p.next()
|
||||||
return &leafNode{t}, nil
|
return &leafNode{t}, nil
|
||||||
default:
|
default:
|
||||||
@@ -253,6 +266,31 @@ func (p *filterParser) parseAtom() (filterNode, error) {
|
|||||||
// Public entry point
|
// Public entry point
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
// parseFilterAST lexes and parses a filter DSL into an AST. Returns (nil, nil)
|
||||||
|
// for an empty or trivial DSL.
|
||||||
|
func parseFilterAST(dsl string) (filterNode, error) {
|
||||||
|
dsl = strings.TrimSpace(dsl)
|
||||||
|
if dsl == "" || dsl == "{}" {
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
toks, err := lexFilter(dsl)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if len(toks) == 0 {
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
p := &filterParser{tokens: toks}
|
||||||
|
node, err := p.parseExpr()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if p.pos != len(p.tokens) {
|
||||||
|
return nil, fmt.Errorf("filter: trailing tokens at position %d", p.pos)
|
||||||
|
}
|
||||||
|
return node, nil
|
||||||
|
}
|
||||||
|
|
||||||
// ParseFilter parses a filter DSL string into a parameterized SQL fragment.
|
// ParseFilter parses a filter DSL string into a parameterized SQL fragment.
|
||||||
//
|
//
|
||||||
// argStart is the 1-based index for the first $N placeholder; this lets the
|
// argStart is the 1-based index for the first $N placeholder; this lets the
|
||||||
@@ -262,25 +300,62 @@ func (p *filterParser) parseAtom() (filterNode, error) {
|
|||||||
// SQL injection is structurally impossible: every user-supplied value is
|
// SQL injection is structurally impossible: every user-supplied value is
|
||||||
// bound as a query parameter ($N), never interpolated into the SQL string.
|
// bound as a query parameter ($N), never interpolated into the SQL string.
|
||||||
func ParseFilter(dsl string, argStart int) (sql string, nextN int, args []any, err error) {
|
func ParseFilter(dsl string, argStart int) (sql string, nextN int, args []any, err error) {
|
||||||
dsl = strings.TrimSpace(dsl)
|
node, err := parseFilterAST(dsl)
|
||||||
if dsl == "" || dsl == "{}" {
|
|
||||||
return "", argStart, nil, nil
|
|
||||||
}
|
|
||||||
toks, err := lexFilter(dsl)
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return "", argStart, nil, err
|
return "", argStart, nil, err
|
||||||
}
|
}
|
||||||
if len(toks) == 0 {
|
if node == nil {
|
||||||
return "", argStart, nil, nil
|
return "", argStart, nil, nil
|
||||||
}
|
}
|
||||||
p := &filterParser{tokens: toks}
|
|
||||||
node, err := p.parseExpr()
|
|
||||||
if err != nil {
|
|
||||||
return "", argStart, nil, err
|
|
||||||
}
|
|
||||||
if p.pos != len(p.tokens) {
|
|
||||||
return "", argStart, nil, fmt.Errorf("filter: trailing tokens at position %d", p.pos)
|
|
||||||
}
|
|
||||||
sql, nextN, args = node.toSQL(argStart, nil)
|
sql, nextN, args = node.toSQL(argStart, nil)
|
||||||
return sql, nextN, args, nil
|
return sql, nextN, args, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// tagUse is a tag referenced by a filter, with whether it was included
|
||||||
|
// (positive) or excluded (negated under an odd number of NOTs).
|
||||||
|
type tagUse struct {
|
||||||
|
tagID uuid.UUID
|
||||||
|
included bool
|
||||||
|
}
|
||||||
|
|
||||||
|
// filterTagUses extracts the distinct tag references in a filter DSL, marking
|
||||||
|
// each as included or excluded. The "untagged" pseudo-token (zero UUID) is
|
||||||
|
// skipped. Returns nil for a filter with no tag terms; an unparseable filter
|
||||||
|
// also yields nil (extraction is best-effort analytics, not validation).
|
||||||
|
func filterTagUses(dsl string) []tagUse {
|
||||||
|
node, err := parseFilterAST(dsl)
|
||||||
|
if err != nil || node == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
seen := make(map[uuid.UUID]bool)
|
||||||
|
collectTagUses(node, true, seen)
|
||||||
|
if len(seen) == 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
uses := make([]tagUse, 0, len(seen))
|
||||||
|
for id, inc := range seen {
|
||||||
|
uses = append(uses, tagUse{tagID: id, included: inc})
|
||||||
|
}
|
||||||
|
return uses
|
||||||
|
}
|
||||||
|
|
||||||
|
// collectTagUses walks the AST, recording each real tag leaf into out keyed by
|
||||||
|
// id. included flips under every NOT, so a tag is "excluded" only when nested
|
||||||
|
// under an odd number of NOTs. A tag appearing under both polarities keeps the
|
||||||
|
// last seen — pathological, but it avoids a duplicate-key insert.
|
||||||
|
func collectTagUses(node filterNode, included bool, out map[uuid.UUID]bool) {
|
||||||
|
switch nd := node.(type) {
|
||||||
|
case *andNode:
|
||||||
|
collectTagUses(nd.left, included, out)
|
||||||
|
collectTagUses(nd.right, included, out)
|
||||||
|
case *orNode:
|
||||||
|
collectTagUses(nd.left, included, out)
|
||||||
|
collectTagUses(nd.right, included, out)
|
||||||
|
case *notNode:
|
||||||
|
collectTagUses(nd.child, !included, out)
|
||||||
|
case *leafNode:
|
||||||
|
if nd.tok.kind == ftkTag && !nd.tok.untagged {
|
||||||
|
out[nd.tok.tagID] = included
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -0,0 +1,97 @@
|
|||||||
|
package postgres
|
||||||
|
|
||||||
|
import (
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/google/uuid"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestParseFilterReview(t *testing.T) {
|
||||||
|
t.Run("r=1 needs review", func(t *testing.T) {
|
||||||
|
sql, n, args, err := ParseFilter("{r=1}", 1)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ParseFilter: %v", err)
|
||||||
|
}
|
||||||
|
if sql != "f.needs_review = $1" {
|
||||||
|
t.Fatalf("sql = %q", sql)
|
||||||
|
}
|
||||||
|
if n != 2 || len(args) != 1 || args[0] != true {
|
||||||
|
t.Fatalf("n=%d args=%v", n, args)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("r=0 reviewed", func(t *testing.T) {
|
||||||
|
sql, _, args, err := ParseFilter("{r=0}", 1)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ParseFilter: %v", err)
|
||||||
|
}
|
||||||
|
if sql != "f.needs_review = $1" || len(args) != 1 || args[0] != false {
|
||||||
|
t.Fatalf("sql=%q args=%v", sql, args)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("combined with mime", func(t *testing.T) {
|
||||||
|
sql, n, args, err := ParseFilter("{r=1,&,m~image/%}", 1)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ParseFilter: %v", err)
|
||||||
|
}
|
||||||
|
if sql != "(f.needs_review = $1 AND mt.name LIKE $2)" {
|
||||||
|
t.Fatalf("sql = %q", sql)
|
||||||
|
}
|
||||||
|
if n != 3 || len(args) != 2 || args[0] != true || args[1] != "image/%" {
|
||||||
|
t.Fatalf("n=%d args=%v", n, args)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("invalid flag rejected", func(t *testing.T) {
|
||||||
|
if _, _, _, err := ParseFilter("{r=2}", 1); err == nil {
|
||||||
|
t.Fatal("expected error for r=2")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFilterTagUses(t *testing.T) {
|
||||||
|
a := uuid.MustParse("11111111-1111-1111-1111-111111111111")
|
||||||
|
b := uuid.MustParse("22222222-2222-2222-2222-222222222222")
|
||||||
|
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
dsl string
|
||||||
|
want map[uuid.UUID]bool // tag → included; absence means "not recorded"
|
||||||
|
}{
|
||||||
|
{"single included", "{t=" + a.String() + "}", map[uuid.UUID]bool{a: true}},
|
||||||
|
{"single excluded", "{!,t=" + a.String() + "}", map[uuid.UUID]bool{a: false}},
|
||||||
|
{"double negation is included", "{!,!,t=" + a.String() + "}", map[uuid.UUID]bool{a: true}},
|
||||||
|
{
|
||||||
|
"and of two included",
|
||||||
|
"{t=" + a.String() + ",&,t=" + b.String() + "}",
|
||||||
|
map[uuid.UUID]bool{a: true, b: true},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"not over a group excludes both",
|
||||||
|
"{!,(,t=" + a.String() + ",|,t=" + b.String() + ",)}",
|
||||||
|
map[uuid.UUID]bool{a: false, b: false},
|
||||||
|
},
|
||||||
|
{"untagged pseudo-token skipped", "{t=" + uuid.Nil.String() + "}", map[uuid.UUID]bool{}},
|
||||||
|
{"mime-only filter records nothing", "{m=3}", map[uuid.UUID]bool{}},
|
||||||
|
{"empty filter", "{}", map[uuid.UUID]bool{}},
|
||||||
|
{"unparseable filter is best-effort nil", "{t=not-a-uuid}", map[uuid.UUID]bool{}},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tc := range tests {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
got := make(map[uuid.UUID]bool)
|
||||||
|
for _, u := range filterTagUses(tc.dsl) {
|
||||||
|
got[u.tagID] = u.included
|
||||||
|
}
|
||||||
|
if len(got) != len(tc.want) {
|
||||||
|
t.Fatalf("got %d uses %v, want %d %v", len(got), got, len(tc.want), tc.want)
|
||||||
|
}
|
||||||
|
for id, inc := range tc.want {
|
||||||
|
if g, ok := got[id]; !ok || g != inc {
|
||||||
|
t.Errorf("tag %s: got (included=%v, present=%v), want included=%v", id, g, ok, inc)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -30,6 +30,8 @@ type poolRow struct {
|
|||||||
CreatorID int16 `db:"creator_id"`
|
CreatorID int16 `db:"creator_id"`
|
||||||
CreatorName string `db:"creator_name"`
|
CreatorName string `db:"creator_name"`
|
||||||
IsPublic bool `db:"is_public"`
|
IsPublic bool `db:"is_public"`
|
||||||
|
SortKey string `db:"sort_key"`
|
||||||
|
SortOrder string `db:"sort_order"`
|
||||||
FileCount int `db:"file_count"`
|
FileCount int `db:"file_count"`
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -68,6 +70,8 @@ func toPool(r poolRow) domain.Pool {
|
|||||||
CreatorID: r.CreatorID,
|
CreatorID: r.CreatorID,
|
||||||
CreatorName: r.CreatorName,
|
CreatorName: r.CreatorName,
|
||||||
IsPublic: r.IsPublic,
|
IsPublic: r.IsPublic,
|
||||||
|
SortKey: r.SortKey,
|
||||||
|
SortOrder: r.SortOrder,
|
||||||
FileCount: r.FileCount,
|
FileCount: r.FileCount,
|
||||||
CreatedAt: domain.UUIDCreatedAt(r.ID),
|
CreatedAt: domain.UUIDCreatedAt(r.ID),
|
||||||
}
|
}
|
||||||
@@ -103,9 +107,15 @@ func toPoolFile(r poolFileRow) domain.PoolFile {
|
|||||||
// Cursor
|
// Cursor
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
// poolFileCursor is the keyset paging cursor for pool files. Which fields are
|
||||||
|
// populated depends on the pool's sort key: Pos for manual (position), Val for
|
||||||
|
// content_datetime (RFC3339Nano) / original_name (the coalesced name); the
|
||||||
|
// "created" sort orders by file id alone, so only FileID matters. FileID is
|
||||||
|
// always the final tiebreak.
|
||||||
type poolFileCursor struct {
|
type poolFileCursor struct {
|
||||||
Position int `json:"p"`
|
Pos int `json:"p,omitempty"`
|
||||||
FileID string `json:"id"`
|
Val string `json:"v,omitempty"`
|
||||||
|
FileID string `json:"id"`
|
||||||
}
|
}
|
||||||
|
|
||||||
func encodePoolCursor(c poolFileCursor) string {
|
func encodePoolCursor(c poolFileCursor) string {
|
||||||
@@ -135,6 +145,7 @@ const poolCountSubquery = `(SELECT pool_id, COUNT(*) AS cnt FROM data.file_pool
|
|||||||
const poolSelectFrom = `
|
const poolSelectFrom = `
|
||||||
SELECT p.id, p.name, p.notes, p.metadata,
|
SELECT p.id, p.name, p.notes, p.metadata,
|
||||||
p.creator_id, u.name AS creator_name, p.is_public,
|
p.creator_id, u.name AS creator_name, p.is_public,
|
||||||
|
p.sort_key, p.sort_order,
|
||||||
COALESCE(fc.cnt, 0) AS file_count
|
COALESCE(fc.cnt, 0) AS file_count
|
||||||
FROM data.pools p
|
FROM data.pools p
|
||||||
JOIN core.users u ON u.id = p.creator_id
|
JOIN core.users u ON u.id = p.creator_id
|
||||||
@@ -147,6 +158,29 @@ func poolSortColumn(s string) string {
|
|||||||
return "p.id" // "created"
|
return "p.id" // "created"
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// poolFileSort maps a pool's stored sort settings to the SQL column expression,
|
||||||
|
// the ORDER BY direction, and the keyset comparison operator used for paging.
|
||||||
|
// The column is chosen so it is never NULL (original_name is coalesced), which
|
||||||
|
// keeps the keyset comparison total.
|
||||||
|
func poolFileSort(sortKey, sortOrder string) (col, dir, cmp string) {
|
||||||
|
dir, cmp = "ASC", ">"
|
||||||
|
if strings.EqualFold(sortOrder, domain.SortOrderDesc) {
|
||||||
|
dir, cmp = "DESC", "<"
|
||||||
|
}
|
||||||
|
switch sortKey {
|
||||||
|
case domain.PoolSortContentDatetime:
|
||||||
|
col = "f.content_datetime"
|
||||||
|
case domain.PoolSortOriginalName:
|
||||||
|
col = "COALESCE(f.original_name, '')"
|
||||||
|
case domain.PoolSortCreated:
|
||||||
|
col = "f.id"
|
||||||
|
default: // manual — the user-arranged sequence; direction does not apply
|
||||||
|
col = "fp.position"
|
||||||
|
dir, cmp = "ASC", ">"
|
||||||
|
}
|
||||||
|
return col, dir, cmp
|
||||||
|
}
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// PoolRepo
|
// PoolRepo
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
@@ -183,6 +217,12 @@ func (r *PoolRepo) List(ctx context.Context, params port.OffsetParams) (*domain.
|
|||||||
args = append(args, "%"+params.Search+"%")
|
args = append(args, "%"+params.Search+"%")
|
||||||
n++
|
n++
|
||||||
}
|
}
|
||||||
|
// Restrict to pools the viewer may see (private-by-default), unless admin.
|
||||||
|
if !params.ViewerIsAdmin {
|
||||||
|
var aclCond string
|
||||||
|
aclCond, n, args = aclVisibilityCond("p", objTypePool, params.ViewerID, n, args)
|
||||||
|
conditions = append(conditions, aclCond)
|
||||||
|
}
|
||||||
|
|
||||||
where := ""
|
where := ""
|
||||||
if len(conditions) > 0 {
|
if len(conditions) > 0 {
|
||||||
@@ -201,6 +241,7 @@ func (r *PoolRepo) List(ctx context.Context, params port.OffsetParams) (*domain.
|
|||||||
query := fmt.Sprintf(`
|
query := fmt.Sprintf(`
|
||||||
SELECT p.id, p.name, p.notes, p.metadata,
|
SELECT p.id, p.name, p.notes, p.metadata,
|
||||||
p.creator_id, u.name AS creator_name, p.is_public,
|
p.creator_id, u.name AS creator_name, p.is_public,
|
||||||
|
p.sort_key, p.sort_order,
|
||||||
COALESCE(fc.cnt, 0) AS file_count,
|
COALESCE(fc.cnt, 0) AS file_count,
|
||||||
COUNT(*) OVER() AS total
|
COUNT(*) OVER() AS total
|
||||||
FROM data.pools p
|
FROM data.pools p
|
||||||
@@ -260,6 +301,16 @@ WHERE p.id = $1`
|
|||||||
return &p, nil
|
return &p, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// RecordView appends a row to activity.pool_views. viewed_at defaults to
|
||||||
|
// statement_timestamp(), so each call records a distinct view in the history.
|
||||||
|
func (r *PoolRepo) RecordView(ctx context.Context, poolID uuid.UUID, userID int16) error {
|
||||||
|
const query = `INSERT INTO activity.pool_views (pool_id, user_id) VALUES ($1, $2)`
|
||||||
|
if _, err := connOrTx(ctx, r.pool).Exec(ctx, query, poolID, userID); err != nil {
|
||||||
|
return fmt.Errorf("PoolRepo.RecordView: %w", err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// Create
|
// Create
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
@@ -273,6 +324,7 @@ WITH ins AS (
|
|||||||
)
|
)
|
||||||
SELECT ins.id, ins.name, ins.notes, ins.metadata,
|
SELECT ins.id, ins.name, ins.notes, ins.metadata,
|
||||||
ins.creator_id, u.name AS creator_name, ins.is_public,
|
ins.creator_id, u.name AS creator_name, ins.is_public,
|
||||||
|
ins.sort_key, ins.sort_order,
|
||||||
0 AS file_count
|
0 AS file_count
|
||||||
FROM ins
|
FROM ins
|
||||||
JOIN core.users u ON u.id = ins.creator_id`
|
JOIN core.users u ON u.id = ins.creator_id`
|
||||||
@@ -306,15 +358,18 @@ func (r *PoolRepo) Update(ctx context.Context, id uuid.UUID, p *domain.Pool) (*d
|
|||||||
const query = `
|
const query = `
|
||||||
WITH upd AS (
|
WITH upd AS (
|
||||||
UPDATE data.pools SET
|
UPDATE data.pools SET
|
||||||
name = $2,
|
name = $2,
|
||||||
notes = $3,
|
notes = $3,
|
||||||
metadata = COALESCE($4, metadata),
|
metadata = COALESCE($4, metadata),
|
||||||
is_public = $5
|
is_public = $5,
|
||||||
|
sort_key = $6,
|
||||||
|
sort_order = $7
|
||||||
WHERE id = $1
|
WHERE id = $1
|
||||||
RETURNING *
|
RETURNING *
|
||||||
)
|
)
|
||||||
SELECT upd.id, upd.name, upd.notes, upd.metadata,
|
SELECT upd.id, upd.name, upd.notes, upd.metadata,
|
||||||
upd.creator_id, u.name AS creator_name, upd.is_public,
|
upd.creator_id, u.name AS creator_name, upd.is_public,
|
||||||
|
upd.sort_key, upd.sort_order,
|
||||||
COALESCE(fc.cnt, 0) AS file_count
|
COALESCE(fc.cnt, 0) AS file_count
|
||||||
FROM upd
|
FROM upd
|
||||||
JOIN core.users u ON u.id = upd.creator_id
|
JOIN core.users u ON u.id = upd.creator_id
|
||||||
@@ -327,7 +382,7 @@ LEFT JOIN (SELECT pool_id, COUNT(*) AS cnt FROM data.file_pool WHERE pool_id = $
|
|||||||
}
|
}
|
||||||
|
|
||||||
q := connOrTx(ctx, r.pool)
|
q := connOrTx(ctx, r.pool)
|
||||||
rows, err := q.Query(ctx, query, id, p.Name, p.Notes, meta, p.IsPublic)
|
rows, err := q.Query(ctx, query, id, p.Name, p.Notes, meta, p.IsPublic, p.SortKey, p.SortOrder)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("PoolRepo.Update: %w", err)
|
return nil, fmt.Errorf("PoolRepo.Update: %w", err)
|
||||||
}
|
}
|
||||||
@@ -396,8 +451,16 @@ func (r *PoolRepo) ListFiles(ctx context.Context, poolID uuid.UUID, params port.
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Cursor condition.
|
// Resolve the pool's sort setting (defaulting to manual position order) into
|
||||||
var orderBy string
|
// the ORDER BY column, direction, and keyset comparison operator.
|
||||||
|
sortKey := params.SortKey
|
||||||
|
if !domain.ValidPoolSortKey(sortKey) {
|
||||||
|
sortKey = domain.PoolSortManual
|
||||||
|
}
|
||||||
|
col, dir, cmp := poolFileSort(sortKey, params.SortOrder)
|
||||||
|
|
||||||
|
// Keyset cursor condition. For "created" the file id is both the sort key and
|
||||||
|
// the tiebreak, so a single comparison suffices; the others compare (col, id).
|
||||||
if params.Cursor != "" {
|
if params.Cursor != "" {
|
||||||
cur, err := decodePoolCursor(params.Cursor)
|
cur, err := decodePoolCursor(params.Cursor)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -407,13 +470,36 @@ func (r *PoolRepo) ListFiles(ctx context.Context, poolID uuid.UUID, params port.
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, domain.ErrValidation
|
return nil, domain.ErrValidation
|
||||||
}
|
}
|
||||||
conds = append(conds, fmt.Sprintf(
|
if sortKey == domain.PoolSortCreated {
|
||||||
"(fp.position > $%d OR (fp.position = $%d AND fp.file_id > $%d))",
|
conds = append(conds, fmt.Sprintf("f.id %s $%d", cmp, n))
|
||||||
n, n, n+1))
|
args = append(args, fileID)
|
||||||
args = append(args, cur.Position, fileID)
|
n++
|
||||||
n += 2
|
} else {
|
||||||
|
conds = append(conds, fmt.Sprintf(
|
||||||
|
"(%s %s $%d OR (%s = $%d AND f.id %s $%d))", col, cmp, n, col, n, cmp, n+1))
|
||||||
|
switch sortKey {
|
||||||
|
case domain.PoolSortContentDatetime:
|
||||||
|
t, err := time.Parse(time.RFC3339Nano, cur.Val)
|
||||||
|
if err != nil {
|
||||||
|
return nil, domain.ErrValidation
|
||||||
|
}
|
||||||
|
args = append(args, t)
|
||||||
|
case domain.PoolSortOriginalName:
|
||||||
|
args = append(args, cur.Val)
|
||||||
|
default: // manual
|
||||||
|
args = append(args, cur.Pos)
|
||||||
|
}
|
||||||
|
args = append(args, fileID)
|
||||||
|
n += 2
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
var orderBy string
|
||||||
|
if sortKey == domain.PoolSortCreated {
|
||||||
|
orderBy = fmt.Sprintf("f.id %s", dir)
|
||||||
|
} else {
|
||||||
|
orderBy = fmt.Sprintf("%s %s, f.id %s", col, dir, dir)
|
||||||
}
|
}
|
||||||
orderBy = "fp.position ASC, fp.file_id ASC"
|
|
||||||
|
|
||||||
where := "WHERE " + strings.Join(conds, " AND ")
|
where := "WHERE " + strings.Join(conds, " AND ")
|
||||||
args = append(args, limit+1)
|
args = append(args, limit+1)
|
||||||
@@ -452,11 +538,21 @@ LIMIT $%d`, fileSelectForPool, where, orderBy, n)
|
|||||||
|
|
||||||
if hasMore && len(collected) > 0 {
|
if hasMore && len(collected) > 0 {
|
||||||
last := collected[len(collected)-1]
|
last := collected[len(collected)-1]
|
||||||
cur := encodePoolCursor(poolFileCursor{
|
cursor := poolFileCursor{FileID: last.ID.String()}
|
||||||
Position: last.Position,
|
switch sortKey {
|
||||||
FileID: last.ID.String(),
|
case domain.PoolSortContentDatetime:
|
||||||
})
|
cursor.Val = last.ContentDatetime.UTC().Format(time.RFC3339Nano)
|
||||||
page.NextCursor = &cur
|
case domain.PoolSortOriginalName:
|
||||||
|
if last.OriginalName != nil {
|
||||||
|
cursor.Val = *last.OriginalName
|
||||||
|
}
|
||||||
|
case domain.PoolSortCreated:
|
||||||
|
// file id alone orders; nothing else to carry
|
||||||
|
default: // manual
|
||||||
|
cursor.Pos = last.Position
|
||||||
|
}
|
||||||
|
enc := encodePoolCursor(cursor)
|
||||||
|
page.NextCursor = &enc
|
||||||
}
|
}
|
||||||
|
|
||||||
// Batch-load tags.
|
// Batch-load tags.
|
||||||
@@ -656,10 +752,54 @@ func (r *PoolRepo) RemoveFiles(ctx context.Context, poolID uuid.UUID, fileIDs []
|
|||||||
// Reorder
|
// Reorder
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
// Reorder replaces the full ordered sequence with positions 1000, 2000, …
|
// Reorder applies the requested order to the pool. Files actually in the pool
|
||||||
// Only file IDs already in the pool are allowed; unknown IDs are silently
|
// are placed in the given order; any pool members the request omitted are kept
|
||||||
// skipped to avoid integrity violations.
|
// and appended in their current order. This makes a partial request (e.g. a
|
||||||
|
// paginated client that only loaded the first pages) reorder the visible prefix
|
||||||
|
// without deleting the rest. Unknown IDs are ignored.
|
||||||
func (r *PoolRepo) Reorder(ctx context.Context, poolID uuid.UUID, fileIDs []uuid.UUID) error {
|
func (r *PoolRepo) Reorder(ctx context.Context, poolID uuid.UUID, fileIDs []uuid.UUID) error {
|
||||||
q := connOrTx(ctx, r.pool)
|
q := connOrTx(ctx, r.pool)
|
||||||
return r.reassignPositions(ctx, q, poolID, fileIDs)
|
|
||||||
}
|
// Current membership, in position order.
|
||||||
|
rows, err := q.Query(ctx,
|
||||||
|
`SELECT file_id FROM data.file_pool WHERE pool_id = $1 ORDER BY position ASC, file_id ASC`, poolID)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("PoolRepo.Reorder fetch: %w", err)
|
||||||
|
}
|
||||||
|
var current []uuid.UUID
|
||||||
|
for rows.Next() {
|
||||||
|
var fid uuid.UUID
|
||||||
|
if err := rows.Scan(&fid); err != nil {
|
||||||
|
rows.Close()
|
||||||
|
return fmt.Errorf("PoolRepo.Reorder scan: %w", err)
|
||||||
|
}
|
||||||
|
current = append(current, fid)
|
||||||
|
}
|
||||||
|
rows.Close()
|
||||||
|
if err := rows.Err(); err != nil {
|
||||||
|
return fmt.Errorf("PoolRepo.Reorder rows: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
inPool := make(map[uuid.UUID]bool, len(current))
|
||||||
|
for _, fid := range current {
|
||||||
|
inPool[fid] = true
|
||||||
|
}
|
||||||
|
|
||||||
|
ordered := make([]uuid.UUID, 0, len(current))
|
||||||
|
placed := make(map[uuid.UUID]bool, len(current))
|
||||||
|
for _, fid := range fileIDs {
|
||||||
|
if inPool[fid] && !placed[fid] {
|
||||||
|
ordered = append(ordered, fid)
|
||||||
|
placed[fid] = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Preserve any members the request did not mention.
|
||||||
|
for _, fid := range current {
|
||||||
|
if !placed[fid] {
|
||||||
|
ordered = append(ordered, fid)
|
||||||
|
placed[fid] = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return r.reassignPositions(ctx, q, poolID, ordered)
|
||||||
|
}
|
||||||
|
|||||||
@@ -11,12 +11,30 @@ import (
|
|||||||
"tanabata/backend/internal/db"
|
"tanabata/backend/internal/db"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
// appName tags every connection as application_name, so the backend's sessions
|
||||||
|
// are identifiable in pg_stat_activity and server logs (and distinguishable from
|
||||||
|
// e.g. goose migrations or a psql shell).
|
||||||
|
const appName = "tanabata-backend"
|
||||||
|
|
||||||
// NewPool creates and validates a *pgxpool.Pool from the given connection URL.
|
// NewPool creates and validates a *pgxpool.Pool from the given connection URL.
|
||||||
// The pool is ready to use; the caller is responsible for closing it.
|
// The pool is ready to use; the caller is responsible for closing it.
|
||||||
func NewPool(ctx context.Context, url string) (*pgxpool.Pool, error) {
|
func NewPool(ctx context.Context, url string) (*pgxpool.Pool, error) {
|
||||||
pool, err := pgxpool.New(ctx, url)
|
cfg, err := pgxpool.ParseConfig(url)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("pgxpool.New: %w", err)
|
return nil, fmt.Errorf("pgxpool.ParseConfig: %w", err)
|
||||||
|
}
|
||||||
|
// Set application_name unless the operator already specified one in the DSN
|
||||||
|
// (or via PGAPPNAME), so an explicit override still wins.
|
||||||
|
if cfg.ConnConfig.RuntimeParams == nil {
|
||||||
|
cfg.ConnConfig.RuntimeParams = map[string]string{}
|
||||||
|
}
|
||||||
|
if cfg.ConnConfig.RuntimeParams["application_name"] == "" {
|
||||||
|
cfg.ConnConfig.RuntimeParams["application_name"] = appName
|
||||||
|
}
|
||||||
|
|
||||||
|
pool, err := pgxpool.NewWithConfig(ctx, cfg)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("pgxpool.NewWithConfig: %w", err)
|
||||||
}
|
}
|
||||||
if err := pool.Ping(ctx); err != nil {
|
if err := pool.Ping(ctx); err != nil {
|
||||||
pool.Close()
|
pool.Close()
|
||||||
@@ -65,3 +83,29 @@ func connOrTx(ctx context.Context, pool *pgxpool.Pool) db.Querier {
|
|||||||
}
|
}
|
||||||
return pool
|
return pool
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Object type IDs as seeded in core.object_types (007_seed_data.sql).
|
||||||
|
const (
|
||||||
|
objTypeFile int16 = 1
|
||||||
|
objTypeTag int16 = 2
|
||||||
|
objTypeCategory int16 = 3
|
||||||
|
objTypePool int16 = 4
|
||||||
|
)
|
||||||
|
|
||||||
|
// aclVisibilityCond returns a SQL boolean fragment that is true when the viewer
|
||||||
|
// may see the row at <alias>.id of the given object type under the
|
||||||
|
// private-by-default model: the row is public, the viewer created it, or the
|
||||||
|
// viewer holds an explicit can_view grant. objectTypeID is a trusted constant
|
||||||
|
// and is inlined; viewerID is bound as $n (referenced twice). Returns the
|
||||||
|
// fragment, the next free parameter index, and the extended args.
|
||||||
|
//
|
||||||
|
// Callers skip this entirely for admins (who bypass ACL).
|
||||||
|
func aclVisibilityCond(alias string, objectTypeID int16, viewerID int16, n int, args []any) (string, int, []any) {
|
||||||
|
cond := fmt.Sprintf(
|
||||||
|
"(%[1]s.is_public OR %[1]s.creator_id = $%[2]d OR EXISTS ("+
|
||||||
|
"SELECT 1 FROM acl.permissions p "+
|
||||||
|
"WHERE p.object_type_id = %[3]d AND p.object_id = %[1]s.id "+
|
||||||
|
"AND p.user_id = $%[2]d AND p.can_view))",
|
||||||
|
alias, n, objectTypeID)
|
||||||
|
return cond, n + 1, append(args, viewerID)
|
||||||
|
}
|
||||||
|
|||||||
@@ -23,16 +23,16 @@ import (
|
|||||||
|
|
||||||
type tagRow struct {
|
type tagRow struct {
|
||||||
ID uuid.UUID `db:"id"`
|
ID uuid.UUID `db:"id"`
|
||||||
Name string `db:"name"`
|
Name string `db:"name"`
|
||||||
Notes *string `db:"notes"`
|
Notes *string `db:"notes"`
|
||||||
Color *string `db:"color"`
|
Color *string `db:"color"`
|
||||||
CategoryID *uuid.UUID `db:"category_id"`
|
CategoryID *uuid.UUID `db:"category_id"`
|
||||||
CategoryName *string `db:"category_name"`
|
CategoryName *string `db:"category_name"`
|
||||||
CategoryColor *string `db:"category_color"`
|
CategoryColor *string `db:"category_color"`
|
||||||
Metadata []byte `db:"metadata"`
|
Metadata []byte `db:"metadata"`
|
||||||
CreatorID int16 `db:"creator_id"`
|
CreatorID int16 `db:"creator_id"`
|
||||||
CreatorName string `db:"creator_name"`
|
CreatorName string `db:"creator_name"`
|
||||||
IsPublic bool `db:"is_public"`
|
IsPublic bool `db:"is_public"`
|
||||||
}
|
}
|
||||||
|
|
||||||
type tagRowWithTotal struct {
|
type tagRowWithTotal struct {
|
||||||
@@ -43,8 +43,8 @@ type tagRowWithTotal struct {
|
|||||||
type tagRuleRow struct {
|
type tagRuleRow struct {
|
||||||
WhenTagID uuid.UUID `db:"when_tag_id"`
|
WhenTagID uuid.UUID `db:"when_tag_id"`
|
||||||
ThenTagID uuid.UUID `db:"then_tag_id"`
|
ThenTagID uuid.UUID `db:"then_tag_id"`
|
||||||
ThenTagName string `db:"then_tag_name"`
|
ThenTagName string `db:"then_tag_name"`
|
||||||
IsActive bool `db:"is_active"`
|
IsActive bool `db:"is_active"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
@@ -155,6 +155,13 @@ func (r *TagRepo) listTags(ctx context.Context, params port.OffsetParams, catego
|
|||||||
}
|
}
|
||||||
sortCol := tagSortColumn(params.Sort)
|
sortCol := tagSortColumn(params.Sort)
|
||||||
|
|
||||||
|
// When sorting by category, break ties within a category by the tag's own
|
||||||
|
// name (same direction), so tags are grouped by category then alphabetical.
|
||||||
|
secondarySort := ""
|
||||||
|
if params.Sort == "category_name" {
|
||||||
|
secondarySort = fmt.Sprintf("t.name %s, ", order)
|
||||||
|
}
|
||||||
|
|
||||||
args := []any{}
|
args := []any{}
|
||||||
n := 1
|
n := 1
|
||||||
var conditions []string
|
var conditions []string
|
||||||
@@ -169,6 +176,12 @@ func (r *TagRepo) listTags(ctx context.Context, params port.OffsetParams, catego
|
|||||||
args = append(args, *categoryID)
|
args = append(args, *categoryID)
|
||||||
n++
|
n++
|
||||||
}
|
}
|
||||||
|
// Restrict to tags the viewer may see (private-by-default), unless admin.
|
||||||
|
if !params.ViewerIsAdmin {
|
||||||
|
var aclCond string
|
||||||
|
aclCond, n, args = aclVisibilityCond("t", objTypeTag, params.ViewerID, n, args)
|
||||||
|
conditions = append(conditions, aclCond)
|
||||||
|
}
|
||||||
|
|
||||||
where := ""
|
where := ""
|
||||||
if len(conditions) > 0 {
|
if len(conditions) > 0 {
|
||||||
@@ -198,8 +211,8 @@ FROM data.tags t
|
|||||||
LEFT JOIN data.categories c ON c.id = t.category_id
|
LEFT JOIN data.categories c ON c.id = t.category_id
|
||||||
JOIN core.users u ON u.id = t.creator_id
|
JOIN core.users u ON u.id = t.creator_id
|
||||||
%s
|
%s
|
||||||
ORDER BY %s %s NULLS LAST, t.id ASC
|
ORDER BY %s %s NULLS LAST, %st.id ASC
|
||||||
LIMIT $%d OFFSET $%d`, where, sortCol, order, n, n+1)
|
LIMIT $%d OFFSET $%d`, where, sortCol, order, secondarySort, n, n+1)
|
||||||
|
|
||||||
args = append(args, limit, offset)
|
args = append(args, limit, offset)
|
||||||
|
|
||||||
@@ -259,7 +272,7 @@ func (r *TagRepo) Create(ctx context.Context, t *domain.Tag) (*domain.Tag, error
|
|||||||
const query = `
|
const query = `
|
||||||
WITH ins AS (
|
WITH ins AS (
|
||||||
INSERT INTO data.tags (name, notes, color, category_id, metadata, creator_id, is_public)
|
INSERT INTO data.tags (name, notes, color, category_id, metadata, creator_id, is_public)
|
||||||
VALUES ($1, $2, $3, $4, $5, $6, $7)
|
VALUES ($1, $2, NULLIF($3, ''), $4, $5, $6, $7)
|
||||||
RETURNING *
|
RETURNING *
|
||||||
)
|
)
|
||||||
SELECT
|
SELECT
|
||||||
@@ -308,7 +321,7 @@ WITH upd AS (
|
|||||||
UPDATE data.tags SET
|
UPDATE data.tags SET
|
||||||
name = $2,
|
name = $2,
|
||||||
notes = $3,
|
notes = $3,
|
||||||
color = $4,
|
color = NULLIF($4, ''),
|
||||||
category_id = $5,
|
category_id = $5,
|
||||||
metadata = COALESCE($6, metadata),
|
metadata = COALESCE($6, metadata),
|
||||||
is_public = $7
|
is_public = $7
|
||||||
@@ -591,10 +604,14 @@ WHERE when_tag_id = $1 AND then_tag_id = $2`
|
|||||||
if !active || !applyToExisting {
|
if !active || !applyToExisting {
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
return r.ApplyToExisting(ctx, whenTagID, thenTagID)
|
||||||
|
}
|
||||||
|
|
||||||
// Retroactively apply the full transitive expansion of thenTagID to all
|
// ApplyToExisting retroactively applies the full transitive expansion of
|
||||||
// files that already carry whenTagID. The recursive CTE walks active rules
|
// thenTagID to all files that already carry whenTagID. The recursive CTE walks
|
||||||
// starting from thenTagID (mirrors the Go expandTagSet BFS).
|
// active rules starting from thenTagID (mirrors the Go expandTagSet BFS), so
|
||||||
|
// inactive downstream rules are not followed. Idempotent via ON CONFLICT.
|
||||||
|
func (r *TagRuleRepo) ApplyToExisting(ctx context.Context, whenTagID, thenTagID uuid.UUID) error {
|
||||||
const retroQuery = `
|
const retroQuery = `
|
||||||
WITH RECURSIVE expansion(tag_id) AS (
|
WITH RECURSIVE expansion(tag_id) AS (
|
||||||
SELECT $2::uuid
|
SELECT $2::uuid
|
||||||
@@ -611,8 +628,9 @@ CROSS JOIN expansion e
|
|||||||
WHERE ft.tag_id = $1
|
WHERE ft.tag_id = $1
|
||||||
ON CONFLICT DO NOTHING`
|
ON CONFLICT DO NOTHING`
|
||||||
|
|
||||||
|
q := connOrTx(ctx, r.pool)
|
||||||
if _, err := q.Exec(ctx, retroQuery, whenTagID, thenTagID); err != nil {
|
if _, err := q.Exec(ctx, retroQuery, whenTagID, thenTagID); err != nil {
|
||||||
return fmt.Errorf("TagRuleRepo.SetActive retroactive apply: %w", err)
|
return fmt.Errorf("TagRuleRepo.ApplyToExisting: %w", err)
|
||||||
}
|
}
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
@@ -631,4 +649,4 @@ WHERE when_tag_id = $1 AND then_tag_id = $2`
|
|||||||
return domain.ErrNotFound
|
return domain.ErrNotFound
|
||||||
}
|
}
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -10,10 +10,10 @@ type ObjectType struct {
|
|||||||
|
|
||||||
// Permission represents a per-object access entry for a user.
|
// Permission represents a per-object access entry for a user.
|
||||||
type Permission struct {
|
type Permission struct {
|
||||||
UserID int16
|
UserID int16
|
||||||
UserName string // denormalized
|
UserName string // denormalized
|
||||||
ObjectTypeID int16
|
ObjectTypeID int16
|
||||||
ObjectID uuid.UUID
|
ObjectID uuid.UUID
|
||||||
CanView bool
|
CanView bool
|
||||||
CanEdit bool
|
CanEdit bool
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -15,14 +15,14 @@ type ActionType struct {
|
|||||||
|
|
||||||
// AuditEntry is a single audit log record.
|
// AuditEntry is a single audit log record.
|
||||||
type AuditEntry struct {
|
type AuditEntry struct {
|
||||||
ID int64
|
ID int64
|
||||||
UserID int16
|
UserID int16
|
||||||
UserName string // denormalized
|
UserName string // denormalized
|
||||||
Action string // action type name, e.g. "file_create"
|
Action string // action type name, e.g. "file_create"
|
||||||
ObjectType *string
|
ObjectType *string
|
||||||
ObjectID *uuid.UUID
|
ObjectID *uuid.UUID
|
||||||
Details json.RawMessage
|
Details json.RawMessage
|
||||||
PerformedAt time.Time
|
PerformedAt time.Time
|
||||||
}
|
}
|
||||||
|
|
||||||
// AuditPage is an offset-based page of audit log entries.
|
// AuditPage is an offset-based page of audit log entries.
|
||||||
|
|||||||
@@ -0,0 +1,18 @@
|
|||||||
|
package domain
|
||||||
|
|
||||||
|
import "github.com/google/uuid"
|
||||||
|
|
||||||
|
// PHashEntry is a file's perceptual hash, the input to duplicate clustering.
|
||||||
|
type PHashEntry struct {
|
||||||
|
ID uuid.UUID
|
||||||
|
PHash int64
|
||||||
|
}
|
||||||
|
|
||||||
|
// DuplicatePair is an unordered pair of files whose perceptual hashes are within
|
||||||
|
// the configured Hamming threshold. FileA < FileB by UUID byte order (canonical),
|
||||||
|
// so a pair is represented exactly once.
|
||||||
|
type DuplicatePair struct {
|
||||||
|
FileA uuid.UUID
|
||||||
|
FileB uuid.UUID
|
||||||
|
Distance int
|
||||||
|
}
|
||||||
@@ -29,6 +29,7 @@ type File struct {
|
|||||||
CreatorName string // denormalized from core.users
|
CreatorName string // denormalized from core.users
|
||||||
IsPublic bool
|
IsPublic bool
|
||||||
IsDeleted bool
|
IsDeleted bool
|
||||||
|
NeedsReview bool // tagging not yet marked done; cleared by an explicit review action
|
||||||
CreatedAt time.Time // extracted from UUID v7 via UUIDCreatedAt
|
CreatedAt time.Time // extracted from UUID v7 via UUIDCreatedAt
|
||||||
Tags []Tag // loaded with the file
|
Tags []Tag // loaded with the file
|
||||||
}
|
}
|
||||||
@@ -49,6 +50,12 @@ type FileListParams struct {
|
|||||||
Filter string // filter DSL expression
|
Filter string // filter DSL expression
|
||||||
Search string // substring match on original_name
|
Search string // substring match on original_name
|
||||||
Trash bool // if true, return only soft-deleted files
|
Trash bool // if true, return only soft-deleted files
|
||||||
|
|
||||||
|
// Visibility — populated by the service from the request context. When
|
||||||
|
// ViewerIsAdmin is false the repository restricts results to files the
|
||||||
|
// viewer may see (public, owned, or explicitly granted).
|
||||||
|
ViewerID int16
|
||||||
|
ViewerIsAdmin bool
|
||||||
}
|
}
|
||||||
|
|
||||||
// FilePage is the result of a cursor-based file listing.
|
// FilePage is the result of a cursor-based file listing.
|
||||||
|
|||||||
@@ -7,6 +7,32 @@ import (
|
|||||||
"github.com/google/uuid"
|
"github.com/google/uuid"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
// Pool file sort keys. PoolSortManual keeps the user-defined order in
|
||||||
|
// file_pool.position; the others sort the pool's files by that file field.
|
||||||
|
const (
|
||||||
|
PoolSortManual = "manual"
|
||||||
|
PoolSortContentDatetime = "content_datetime"
|
||||||
|
PoolSortCreated = "created"
|
||||||
|
PoolSortOriginalName = "original_name"
|
||||||
|
|
||||||
|
SortOrderAsc = "asc"
|
||||||
|
SortOrderDesc = "desc"
|
||||||
|
)
|
||||||
|
|
||||||
|
// ValidPoolSortKey reports whether s is an accepted pool sort key.
|
||||||
|
func ValidPoolSortKey(s string) bool {
|
||||||
|
switch s {
|
||||||
|
case PoolSortManual, PoolSortContentDatetime, PoolSortCreated, PoolSortOriginalName:
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
// ValidSortOrder reports whether s is an accepted sort direction.
|
||||||
|
func ValidSortOrder(s string) bool {
|
||||||
|
return s == SortOrderAsc || s == SortOrderDesc
|
||||||
|
}
|
||||||
|
|
||||||
// Pool is an ordered collection of files.
|
// Pool is an ordered collection of files.
|
||||||
type Pool struct {
|
type Pool struct {
|
||||||
ID uuid.UUID
|
ID uuid.UUID
|
||||||
@@ -16,8 +42,13 @@ type Pool struct {
|
|||||||
CreatorID int16
|
CreatorID int16
|
||||||
CreatorName string // denormalized
|
CreatorName string // denormalized
|
||||||
IsPublic bool
|
IsPublic bool
|
||||||
FileCount int
|
// SortKey / SortOrder control how the pool's files are ordered. When SortKey
|
||||||
CreatedAt time.Time // extracted from UUID v7 via UUIDCreatedAt
|
// is PoolSortManual, files follow the manual position order and can be
|
||||||
|
// reordered; otherwise they are sorted automatically and reordering is a no-op.
|
||||||
|
SortKey string
|
||||||
|
SortOrder string
|
||||||
|
FileCount int
|
||||||
|
CreatedAt time.Time // extracted from UUID v7 via UUIDCreatedAt
|
||||||
}
|
}
|
||||||
|
|
||||||
// PoolFile is a File with its ordering position within a pool.
|
// PoolFile is a File with its ordering position within a pool.
|
||||||
|
|||||||
@@ -143,4 +143,4 @@ func (h *ACLHandler) SetPermissions(c *gin.Context) {
|
|||||||
out[i] = toPermissionJSON(p)
|
out[i] = toPermissionJSON(p)
|
||||||
}
|
}
|
||||||
respondJSON(c, http.StatusOK, out)
|
respondJSON(c, http.StatusOK, out)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -117,4 +117,4 @@ func (h *AuditHandler) List(c *gin.Context) {
|
|||||||
"offset": page.Offset,
|
"offset": page.Offset,
|
||||||
"limit": page.Limit,
|
"limit": page.Limit,
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -232,4 +232,4 @@ func (h *CategoryHandler) ListTags(c *gin.Context) {
|
|||||||
"offset": page.Offset,
|
"offset": page.Offset,
|
||||||
"limit": page.Limit,
|
"limit": page.Limit,
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,126 @@
|
|||||||
|
package handler
|
||||||
|
|
||||||
|
import (
|
||||||
|
"net/http"
|
||||||
|
"strconv"
|
||||||
|
|
||||||
|
"github.com/gin-gonic/gin"
|
||||||
|
|
||||||
|
"tanabata/backend/internal/domain"
|
||||||
|
"tanabata/backend/internal/service"
|
||||||
|
)
|
||||||
|
|
||||||
|
// DuplicateHandler handles the /files/duplicates endpoints.
|
||||||
|
type DuplicateHandler struct {
|
||||||
|
dupSvc *service.DuplicateService
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewDuplicateHandler creates a DuplicateHandler.
|
||||||
|
func NewDuplicateHandler(dupSvc *service.DuplicateService) *DuplicateHandler {
|
||||||
|
return &DuplicateHandler{dupSvc: dupSvc}
|
||||||
|
}
|
||||||
|
|
||||||
|
// List handles GET /files/duplicates — an offset-paginated list of duplicate
|
||||||
|
// clusters, each a group of files within the perceptual-hash threshold.
|
||||||
|
func (h *DuplicateHandler) List(c *gin.Context) {
|
||||||
|
limit, offset := 20, 0
|
||||||
|
if n, err := strconv.Atoi(c.Query("limit")); err == nil {
|
||||||
|
limit = n
|
||||||
|
}
|
||||||
|
if n, err := strconv.Atoi(c.Query("offset")); err == nil {
|
||||||
|
offset = n
|
||||||
|
}
|
||||||
|
if limit < 1 {
|
||||||
|
limit = 1
|
||||||
|
}
|
||||||
|
if limit > 50 {
|
||||||
|
limit = 50
|
||||||
|
}
|
||||||
|
if offset < 0 {
|
||||||
|
offset = 0
|
||||||
|
}
|
||||||
|
|
||||||
|
clusters, total, err := h.dupSvc.Clusters(c.Request.Context(), limit, offset)
|
||||||
|
if err != nil {
|
||||||
|
respondError(c, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
items := make([]gin.H, len(clusters))
|
||||||
|
for i, cl := range clusters {
|
||||||
|
fs := make([]fileJSON, len(cl.Files))
|
||||||
|
for j, f := range cl.Files {
|
||||||
|
fs[j] = toFileJSON(f)
|
||||||
|
}
|
||||||
|
dists := make([]gin.H, len(cl.Distances))
|
||||||
|
for j, d := range cl.Distances {
|
||||||
|
dists[j] = gin.H{"a": d.A, "b": d.B, "distance": d.Distance}
|
||||||
|
}
|
||||||
|
items[i] = gin.H{"files": fs, "distances": dists}
|
||||||
|
}
|
||||||
|
respondJSON(c, http.StatusOK, gin.H{
|
||||||
|
"items": items,
|
||||||
|
"total": total,
|
||||||
|
"limit": limit,
|
||||||
|
"offset": offset,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// Dismiss handles POST /files/duplicates/dismiss — mark a pair "not a duplicate".
|
||||||
|
func (h *DuplicateHandler) Dismiss(c *gin.Context) {
|
||||||
|
var body struct {
|
||||||
|
FileIDA string `json:"file_id_a" binding:"required"`
|
||||||
|
FileIDB string `json:"file_id_b" binding:"required"`
|
||||||
|
}
|
||||||
|
if err := c.ShouldBindJSON(&body); err != nil {
|
||||||
|
respondError(c, domain.ErrValidation)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
ids, err := parseUUIDs([]string{body.FileIDA, body.FileIDB})
|
||||||
|
if err != nil {
|
||||||
|
respondError(c, domain.ErrValidation)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err := h.dupSvc.Dismiss(c.Request.Context(), ids[0], ids[1]); err != nil {
|
||||||
|
respondError(c, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
c.Status(http.StatusNoContent)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Resolve handles POST /files/duplicates/resolve — merge a duplicate pair,
|
||||||
|
// keeping one file and folding the chosen fields in from the other. Returns the
|
||||||
|
// updated survivor. delete_discarded defaults to true.
|
||||||
|
func (h *DuplicateHandler) Resolve(c *gin.Context) {
|
||||||
|
var body struct {
|
||||||
|
Keep string `json:"keep" binding:"required"`
|
||||||
|
Discard string `json:"discard" binding:"required"`
|
||||||
|
Fields service.MergeFields `json:"fields"`
|
||||||
|
DeleteDiscarded *bool `json:"delete_discarded"`
|
||||||
|
}
|
||||||
|
if err := c.ShouldBindJSON(&body); err != nil {
|
||||||
|
respondError(c, domain.ErrValidation)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
ids, err := parseUUIDs([]string{body.Keep, body.Discard})
|
||||||
|
if err != nil {
|
||||||
|
respondError(c, domain.ErrValidation)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
del := true
|
||||||
|
if body.DeleteDiscarded != nil {
|
||||||
|
del = *body.DeleteDiscarded
|
||||||
|
}
|
||||||
|
f, err := h.dupSvc.Resolve(c.Request.Context(), service.MergeSpec{
|
||||||
|
Keep: ids[0],
|
||||||
|
Discard: ids[1],
|
||||||
|
Fields: body.Fields,
|
||||||
|
DeleteDiscarded: del,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
respondError(c, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
respondJSON(c, http.StatusOK, toFileJSON(*f))
|
||||||
|
}
|
||||||
@@ -22,13 +22,14 @@ import (
|
|||||||
type FileHandler struct {
|
type FileHandler struct {
|
||||||
fileSvc *service.FileService
|
fileSvc *service.FileService
|
||||||
tagSvc *service.TagService
|
tagSvc *service.TagService
|
||||||
|
authSvc *service.AuthService
|
||||||
maxUploadBytes int64
|
maxUploadBytes int64
|
||||||
}
|
}
|
||||||
|
|
||||||
// NewFileHandler creates a FileHandler. maxUploadBytes caps the size of an
|
// NewFileHandler creates a FileHandler. maxUploadBytes caps the size of an
|
||||||
// uploaded or replacement file.
|
// uploaded or replacement file. authSvc mints content tokens for media URLs.
|
||||||
func NewFileHandler(fileSvc *service.FileService, tagSvc *service.TagService, maxUploadBytes int64) *FileHandler {
|
func NewFileHandler(fileSvc *service.FileService, tagSvc *service.TagService, authSvc *service.AuthService, maxUploadBytes int64) *FileHandler {
|
||||||
return &FileHandler{fileSvc: fileSvc, tagSvc: tagSvc, maxUploadBytes: maxUploadBytes}
|
return &FileHandler{fileSvc: fileSvc, tagSvc: tagSvc, authSvc: authSvc, maxUploadBytes: maxUploadBytes}
|
||||||
}
|
}
|
||||||
|
|
||||||
// formFileLimited reads the "file" multipart field while bounding how many bytes
|
// formFileLimited reads the "file" multipart field while bounding how many bytes
|
||||||
@@ -83,22 +84,23 @@ type fileJSON struct {
|
|||||||
CreatorName string `json:"creator_name"`
|
CreatorName string `json:"creator_name"`
|
||||||
IsPublic bool `json:"is_public"`
|
IsPublic bool `json:"is_public"`
|
||||||
IsDeleted bool `json:"is_deleted"`
|
IsDeleted bool `json:"is_deleted"`
|
||||||
|
NeedsReview bool `json:"needs_review"`
|
||||||
CreatedAt string `json:"created_at"`
|
CreatedAt string `json:"created_at"`
|
||||||
Tags []tagJSON `json:"tags"`
|
Tags []tagJSON `json:"tags"`
|
||||||
}
|
}
|
||||||
|
|
||||||
func toTagJSON(t domain.Tag) tagJSON {
|
func toTagJSON(t domain.Tag) tagJSON {
|
||||||
j := tagJSON{
|
j := tagJSON{
|
||||||
ID: t.ID.String(),
|
ID: t.ID.String(),
|
||||||
Name: t.Name,
|
Name: t.Name,
|
||||||
Notes: t.Notes,
|
Notes: t.Notes,
|
||||||
Color: t.Color,
|
Color: t.Color,
|
||||||
CategoryName: t.CategoryName,
|
CategoryName: t.CategoryName,
|
||||||
CategoryColor: t.CategoryColor,
|
CategoryColor: t.CategoryColor,
|
||||||
CreatorID: t.CreatorID,
|
CreatorID: t.CreatorID,
|
||||||
CreatorName: t.CreatorName,
|
CreatorName: t.CreatorName,
|
||||||
IsPublic: t.IsPublic,
|
IsPublic: t.IsPublic,
|
||||||
CreatedAt: t.CreatedAt.Format(time.RFC3339),
|
CreatedAt: t.CreatedAt.Format(time.RFC3339),
|
||||||
}
|
}
|
||||||
if t.CategoryID != nil {
|
if t.CategoryID != nil {
|
||||||
s := t.CategoryID.String()
|
s := t.CategoryID.String()
|
||||||
@@ -130,6 +132,7 @@ func toFileJSON(f domain.File) fileJSON {
|
|||||||
CreatorName: f.CreatorName,
|
CreatorName: f.CreatorName,
|
||||||
IsPublic: f.IsPublic,
|
IsPublic: f.IsPublic,
|
||||||
IsDeleted: f.IsDeleted,
|
IsDeleted: f.IsDeleted,
|
||||||
|
NeedsReview: f.NeedsReview,
|
||||||
CreatedAt: f.CreatedAt.Format(time.RFC3339),
|
CreatedAt: f.CreatedAt.Format(time.RFC3339),
|
||||||
Tags: tags,
|
Tags: tags,
|
||||||
}
|
}
|
||||||
@@ -300,6 +303,25 @@ func (h *FileHandler) GetMeta(c *gin.Context) {
|
|||||||
respondJSON(c, http.StatusOK, toFileJSON(*f))
|
respondJSON(c, http.StatusOK, toFileJSON(*f))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// POST /files/:id/views
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
// RecordView logs that the current user viewed the file (activity.file_views).
|
||||||
|
func (h *FileHandler) RecordView(c *gin.Context) {
|
||||||
|
id, ok := parseFileID(c)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := h.fileSvc.RecordView(c.Request.Context(), id); err != nil {
|
||||||
|
respondError(c, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.Status(http.StatusNoContent)
|
||||||
|
}
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// PATCH /files/:id
|
// PATCH /files/:id
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
@@ -364,6 +386,38 @@ func (h *FileHandler) SoftDelete(c *gin.Context) {
|
|||||||
c.Status(http.StatusNoContent)
|
c.Status(http.StatusNoContent)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// POST /files/:id/content-token
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
// CreateContentToken mints a short-lived, single-file capability token the
|
||||||
|
// client can put in a content URL's access_token query parameter to open or
|
||||||
|
// stream the original by link (e.g. a long video in a new tab) without the URL
|
||||||
|
// dying when the 15-minute access token expires. It first enforces view
|
||||||
|
// permission via fileSvc.Get, so a token is only issued for a file the caller
|
||||||
|
// may actually read.
|
||||||
|
func (h *FileHandler) CreateContentToken(c *gin.Context) {
|
||||||
|
id, ok := parseFileID(c)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// Authorize (and confirm existence) the same way content serving does.
|
||||||
|
if _, err := h.fileSvc.Get(c.Request.Context(), id); err != nil {
|
||||||
|
respondError(c, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
userID, isAdmin, _ := domain.UserFromContext(c.Request.Context())
|
||||||
|
token, expiresIn, err := h.authSvc.GenerateContentToken(id.String(), userID, isAdmin)
|
||||||
|
if err != nil {
|
||||||
|
respondError(c, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.JSON(http.StatusOK, gin.H{"token": token, "expires_in": expiresIn})
|
||||||
|
}
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// GET /files/:id/content
|
// GET /files/:id/content
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
@@ -382,9 +436,26 @@ func (h *FileHandler) GetContent(c *gin.Context) {
|
|||||||
defer res.Body.Close()
|
defer res.Body.Close()
|
||||||
|
|
||||||
c.Header("Content-Type", res.MIMEType)
|
c.Header("Content-Type", res.MIMEType)
|
||||||
|
c.Header("Cache-Control", "private, max-age=3600")
|
||||||
|
// Default to attachment (download); ?inline=1 serves it for in-tab viewing.
|
||||||
|
disposition := "attachment"
|
||||||
|
if c.Query("inline") == "1" {
|
||||||
|
disposition = "inline"
|
||||||
|
}
|
||||||
|
name := ""
|
||||||
if res.OriginalName != nil {
|
if res.OriginalName != nil {
|
||||||
|
name = *res.OriginalName
|
||||||
c.Header("Content-Disposition",
|
c.Header("Content-Disposition",
|
||||||
fmt.Sprintf("attachment; filename=%q", *res.OriginalName))
|
fmt.Sprintf("%s; filename=%q", disposition, name))
|
||||||
|
}
|
||||||
|
|
||||||
|
// Serve with byte-range support when the body is seekable (it is for the
|
||||||
|
// disk store): http.ServeContent advertises Accept-Ranges and answers Range
|
||||||
|
// requests with 206 Partial Content, which is what lets the browser scrub and
|
||||||
|
// seek within audio/video. Fall back to a plain stream otherwise.
|
||||||
|
if seeker, ok := res.Body.(io.ReadSeeker); ok {
|
||||||
|
http.ServeContent(c.Writer, c.Request, name, time.Time{}, seeker)
|
||||||
|
return
|
||||||
}
|
}
|
||||||
c.Status(http.StatusOK)
|
c.Status(http.StatusOK)
|
||||||
io.Copy(c.Writer, res.Body) //nolint:errcheck
|
io.Copy(c.Writer, res.Body) //nolint:errcheck
|
||||||
@@ -457,6 +528,7 @@ func (h *FileHandler) GetThumbnail(c *gin.Context) {
|
|||||||
defer rc.Close()
|
defer rc.Close()
|
||||||
|
|
||||||
c.Header("Content-Type", "image/jpeg")
|
c.Header("Content-Type", "image/jpeg")
|
||||||
|
c.Header("Cache-Control", "private, max-age=3600")
|
||||||
c.Status(http.StatusOK)
|
c.Status(http.StatusOK)
|
||||||
io.Copy(c.Writer, rc) //nolint:errcheck
|
io.Copy(c.Writer, rc) //nolint:errcheck
|
||||||
}
|
}
|
||||||
@@ -479,6 +551,7 @@ func (h *FileHandler) GetPreview(c *gin.Context) {
|
|||||||
defer rc.Close()
|
defer rc.Close()
|
||||||
|
|
||||||
c.Header("Content-Type", "image/jpeg")
|
c.Header("Content-Type", "image/jpeg")
|
||||||
|
c.Header("Cache-Control", "private, max-age=3600")
|
||||||
c.Status(http.StatusOK)
|
c.Status(http.StatusOK)
|
||||||
io.Copy(c.Writer, rc) //nolint:errcheck
|
io.Copy(c.Writer, rc) //nolint:errcheck
|
||||||
}
|
}
|
||||||
@@ -590,6 +663,33 @@ func (h *FileHandler) BulkDelete(c *gin.Context) {
|
|||||||
c.Status(http.StatusNoContent)
|
c.Status(http.StatusNoContent)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// BulkReview sets the review status on one or more files. A single-file toggle
|
||||||
|
// is just a one-element file_ids array. Files the caller cannot edit are
|
||||||
|
// silently skipped (handled in the service).
|
||||||
|
func (h *FileHandler) BulkReview(c *gin.Context) {
|
||||||
|
var body struct {
|
||||||
|
FileIDs []string `json:"file_ids" binding:"required"`
|
||||||
|
NeedsReview *bool `json:"needs_review" binding:"required"`
|
||||||
|
}
|
||||||
|
if err := c.ShouldBindJSON(&body); err != nil || body.NeedsReview == nil {
|
||||||
|
respondError(c, domain.ErrValidation)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
fileIDs, err := parseUUIDs(body.FileIDs)
|
||||||
|
if err != nil {
|
||||||
|
respondError(c, domain.ErrValidation)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := h.fileSvc.SetNeedsReview(c.Request.Context(), fileIDs, *body.NeedsReview); err != nil {
|
||||||
|
respondError(c, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.Status(http.StatusNoContent)
|
||||||
|
}
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// POST /files/bulk/common-tags
|
// POST /files/bulk/common-tags
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
@@ -646,13 +746,36 @@ func (h *FileHandler) Import(c *gin.Context) {
|
|||||||
// Body is optional; ignore bind errors.
|
// Body is optional; ignore bind errors.
|
||||||
_ = c.ShouldBindJSON(&body)
|
_ = c.ShouldBindJSON(&body)
|
||||||
|
|
||||||
result, err := h.fileSvc.Import(c.Request.Context(), body.Path)
|
// Stream progress as newline-delimited JSON so the client can render a live
|
||||||
if err != nil {
|
// progress bar and per-file status. Headers are deferred until the first
|
||||||
respondError(c, err)
|
// event, so a validation error (bad path, import disabled) raised before any
|
||||||
return
|
// file is touched can still be returned as a normal JSON error response.
|
||||||
|
flusher, canFlush := c.Writer.(http.Flusher)
|
||||||
|
started := false
|
||||||
|
enc := json.NewEncoder(c.Writer)
|
||||||
|
|
||||||
|
emit := func(ev service.ImportEvent) {
|
||||||
|
if !started {
|
||||||
|
c.Header("Content-Type", "application/x-ndjson")
|
||||||
|
c.Header("Cache-Control", "no-cache")
|
||||||
|
c.Header("X-Accel-Buffering", "no") // don't let a proxy buffer the stream
|
||||||
|
c.Writer.WriteHeader(http.StatusOK)
|
||||||
|
started = true
|
||||||
|
}
|
||||||
|
_ = enc.Encode(ev) // appends a newline
|
||||||
|
if canFlush {
|
||||||
|
flusher.Flush()
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
respondJSON(c, http.StatusOK, result)
|
if _, err := h.fileSvc.Import(c.Request.Context(), body.Path, emit); err != nil {
|
||||||
|
if !started {
|
||||||
|
respondError(c, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
// Headers already sent; surface the failure as a terminal stream event.
|
||||||
|
emit(service.ImportEvent{Type: "error", Reason: err.Error()})
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
@@ -669,4 +792,4 @@ func parseUUIDs(strs []string) ([]uuid.UUID, error) {
|
|||||||
ids = append(ids, id)
|
ids = append(ids, id)
|
||||||
}
|
}
|
||||||
return ids, nil
|
return ids, nil
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -5,6 +5,7 @@ import (
|
|||||||
"strings"
|
"strings"
|
||||||
|
|
||||||
"github.com/gin-gonic/gin"
|
"github.com/gin-gonic/gin"
|
||||||
|
"github.com/google/uuid"
|
||||||
|
|
||||||
"tanabata/backend/internal/domain"
|
"tanabata/backend/internal/domain"
|
||||||
"tanabata/backend/internal/service"
|
"tanabata/backend/internal/service"
|
||||||
@@ -24,8 +25,8 @@ func NewAuthMiddleware(authSvc *service.AuthService) *AuthMiddleware {
|
|||||||
// On success it calls c.Next(); on failure it aborts with 401 JSON.
|
// On success it calls c.Next(); on failure it aborts with 401 JSON.
|
||||||
func (m *AuthMiddleware) Handle() gin.HandlerFunc {
|
func (m *AuthMiddleware) Handle() gin.HandlerFunc {
|
||||||
return func(c *gin.Context) {
|
return func(c *gin.Context) {
|
||||||
raw := c.GetHeader("Authorization")
|
token := bearerToken(c)
|
||||||
if !strings.HasPrefix(raw, "Bearer ") {
|
if token == "" {
|
||||||
c.JSON(http.StatusUnauthorized, errorBody{
|
c.JSON(http.StatusUnauthorized, errorBody{
|
||||||
Code: domain.ErrUnauthorized.Code(),
|
Code: domain.ErrUnauthorized.Code(),
|
||||||
Message: "authorization header missing or malformed",
|
Message: "authorization header missing or malformed",
|
||||||
@@ -33,7 +34,6 @@ func (m *AuthMiddleware) Handle() gin.HandlerFunc {
|
|||||||
c.Abort()
|
c.Abort()
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
token := strings.TrimPrefix(raw, "Bearer ")
|
|
||||||
|
|
||||||
claims, err := m.authSvc.ValidateAccessToken(c.Request.Context(), token)
|
claims, err := m.authSvc.ValidateAccessToken(c.Request.Context(), token)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -50,3 +50,67 @@ func (m *AuthMiddleware) Handle() gin.HandlerFunc {
|
|||||||
c.Next()
|
c.Next()
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// HandleContent authenticates a file-content GET, accepting either a normal
|
||||||
|
// access token or a content token scoped (by its fid claim) to the :id in the
|
||||||
|
// path. The content token is what keeps a long media stream playing after the
|
||||||
|
// short access token would have expired. View permission is still enforced in
|
||||||
|
// the handler against the resolved user, so a content token only widens *when*
|
||||||
|
// a file may be read by URL, never *which* files.
|
||||||
|
func (m *AuthMiddleware) HandleContent() gin.HandlerFunc {
|
||||||
|
return func(c *gin.Context) {
|
||||||
|
token := bearerToken(c)
|
||||||
|
if token == "" {
|
||||||
|
contentUnauthorized(c)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// A regular access token grants access to everything as usual.
|
||||||
|
if claims, err := m.authSvc.ValidateAccessToken(c.Request.Context(), token); err == nil {
|
||||||
|
ctx := domain.WithUser(c.Request.Context(), claims.UserID, claims.IsAdmin, claims.SessionID)
|
||||||
|
c.Request = c.Request.WithContext(ctx)
|
||||||
|
c.Next()
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// Otherwise accept a content token minted for exactly this file. Normalise
|
||||||
|
// the path id to canonical form so it matches the minted fid claim.
|
||||||
|
id, err := uuid.Parse(c.Param("id"))
|
||||||
|
if err != nil {
|
||||||
|
contentUnauthorized(c)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
claims, err := m.authSvc.ValidateContentToken(token, id.String())
|
||||||
|
if err != nil {
|
||||||
|
contentUnauthorized(c)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
// A content token carries no session (sid 0); it is session-independent.
|
||||||
|
ctx := domain.WithUser(c.Request.Context(), claims.UserID, claims.IsAdmin, claims.SessionID)
|
||||||
|
c.Request = c.Request.WithContext(ctx)
|
||||||
|
c.Next()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func contentUnauthorized(c *gin.Context) {
|
||||||
|
c.JSON(http.StatusUnauthorized, errorBody{
|
||||||
|
Code: domain.ErrUnauthorized.Code(),
|
||||||
|
Message: "invalid or expired token",
|
||||||
|
})
|
||||||
|
c.Abort()
|
||||||
|
}
|
||||||
|
|
||||||
|
// bearerToken extracts the access token from the Authorization header. As a
|
||||||
|
// fallback it accepts an ?access_token= query parameter, but only for GET
|
||||||
|
// requests — this lets the browser open media (e.g. /files/{id}/content) via a
|
||||||
|
// plain link/new tab, where it can't send the header, without allowing a crafted
|
||||||
|
// link to drive a state-changing request.
|
||||||
|
func bearerToken(c *gin.Context) string {
|
||||||
|
if raw := c.GetHeader("Authorization"); strings.HasPrefix(raw, "Bearer ") {
|
||||||
|
return strings.TrimPrefix(raw, "Bearer ")
|
||||||
|
}
|
||||||
|
if c.Request.Method == http.MethodGet {
|
||||||
|
return c.Query("access_token")
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|||||||
@@ -34,6 +34,8 @@ type poolJSON struct {
|
|||||||
CreatorID int16 `json:"creator_id"`
|
CreatorID int16 `json:"creator_id"`
|
||||||
CreatorName string `json:"creator_name"`
|
CreatorName string `json:"creator_name"`
|
||||||
IsPublic bool `json:"is_public"`
|
IsPublic bool `json:"is_public"`
|
||||||
|
SortKey string `json:"sort_key"`
|
||||||
|
SortOrder string `json:"sort_order"`
|
||||||
FileCount int `json:"file_count"`
|
FileCount int `json:"file_count"`
|
||||||
CreatedAt string `json:"created_at"`
|
CreatedAt string `json:"created_at"`
|
||||||
}
|
}
|
||||||
@@ -51,6 +53,8 @@ func toPoolJSON(p domain.Pool) poolJSON {
|
|||||||
CreatorID: p.CreatorID,
|
CreatorID: p.CreatorID,
|
||||||
CreatorName: p.CreatorName,
|
CreatorName: p.CreatorName,
|
||||||
IsPublic: p.IsPublic,
|
IsPublic: p.IsPublic,
|
||||||
|
SortKey: p.SortKey,
|
||||||
|
SortOrder: p.SortOrder,
|
||||||
FileCount: p.FileCount,
|
FileCount: p.FileCount,
|
||||||
CreatedAt: p.CreatedAt.UTC().Format(time.RFC3339),
|
CreatedAt: p.CreatedAt.UTC().Format(time.RFC3339),
|
||||||
}
|
}
|
||||||
@@ -160,6 +164,25 @@ func (h *PoolHandler) Get(c *gin.Context) {
|
|||||||
respondJSON(c, http.StatusOK, toPoolJSON(*p))
|
respondJSON(c, http.StatusOK, toPoolJSON(*p))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// POST /pools/:pool_id/views
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
// RecordView logs that the current user viewed the pool (activity.pool_views).
|
||||||
|
func (h *PoolHandler) RecordView(c *gin.Context) {
|
||||||
|
id, ok := parsePoolID(c)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := h.poolSvc.RecordView(c.Request.Context(), id); err != nil {
|
||||||
|
respondError(c, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
c.Status(http.StatusNoContent)
|
||||||
|
}
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// PATCH /pools/:pool_id
|
// PATCH /pools/:pool_id
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
@@ -195,6 +218,16 @@ func (h *PoolHandler) Update(c *gin.Context) {
|
|||||||
params.IsPublic = &b
|
params.IsPublic = &b
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
if v, ok := raw["sort_key"]; ok {
|
||||||
|
if s, ok := v.(string); ok {
|
||||||
|
params.SortKey = &s
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if v, ok := raw["sort_order"]; ok {
|
||||||
|
if s, ok := v.(string); ok {
|
||||||
|
params.SortOrder = &s
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
updated, err := h.poolSvc.Update(c.Request.Context(), id, params)
|
updated, err := h.poolSvc.Update(c.Request.Context(), id, params)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -353,4 +386,4 @@ func (h *PoolHandler) Reorder(c *gin.Context) {
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
c.Status(http.StatusNoContent)
|
c.Status(http.StatusNoContent)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
package handler
|
package handler
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"fmt"
|
||||||
"net/http"
|
"net/http"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
@@ -25,16 +26,28 @@ func NewRouter(
|
|||||||
auth *AuthMiddleware,
|
auth *AuthMiddleware,
|
||||||
authHandler *AuthHandler,
|
authHandler *AuthHandler,
|
||||||
fileHandler *FileHandler,
|
fileHandler *FileHandler,
|
||||||
|
duplicateHandler *DuplicateHandler,
|
||||||
tagHandler *TagHandler,
|
tagHandler *TagHandler,
|
||||||
categoryHandler *CategoryHandler,
|
categoryHandler *CategoryHandler,
|
||||||
poolHandler *PoolHandler,
|
poolHandler *PoolHandler,
|
||||||
userHandler *UserHandler,
|
userHandler *UserHandler,
|
||||||
aclHandler *ACLHandler,
|
aclHandler *ACLHandler,
|
||||||
auditHandler *AuditHandler,
|
auditHandler *AuditHandler,
|
||||||
) *gin.Engine {
|
staticDir string,
|
||||||
|
trustedProxies []string,
|
||||||
|
) (*gin.Engine, error) {
|
||||||
r := gin.New()
|
r := gin.New()
|
||||||
r.Use(gin.Logger(), gin.Recovery(), securityHeaders())
|
r.Use(gin.Logger(), gin.Recovery(), securityHeaders())
|
||||||
|
|
||||||
|
// Behind a reverse proxy the client's real IP arrives in X-Forwarded-For.
|
||||||
|
// Trust only the proxy hop(s) so c.ClientIP() — used by the auth rate
|
||||||
|
// limiter — reflects the real client and can't be spoofed by a forged
|
||||||
|
// header from a direct caller. An empty list trusts no proxy (ClientIP is
|
||||||
|
// the immediate peer).
|
||||||
|
if err := r.SetTrustedProxies(trustedProxies); err != nil {
|
||||||
|
return nil, fmt.Errorf("configure trusted proxies: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
// Health check — no auth required.
|
// Health check — no auth required.
|
||||||
r.GET("/health", func(c *gin.Context) {
|
r.GET("/health", func(c *gin.Context) {
|
||||||
c.JSON(http.StatusOK, gin.H{"status": "ok"})
|
c.JSON(http.StatusOK, gin.H{"status": "ok"})
|
||||||
@@ -68,9 +81,14 @@ func NewRouter(
|
|||||||
files.GET("", fileHandler.List)
|
files.GET("", fileHandler.List)
|
||||||
files.POST("", fileHandler.Upload)
|
files.POST("", fileHandler.Upload)
|
||||||
|
|
||||||
// Bulk + import routes registered before /:id to prevent param collision.
|
// Bulk + import + duplicates routes registered before /:id to prevent
|
||||||
|
// param collision (e.g. "duplicates" being captured as :id).
|
||||||
|
files.GET("/duplicates", duplicateHandler.List)
|
||||||
|
files.POST("/duplicates/dismiss", duplicateHandler.Dismiss)
|
||||||
|
files.POST("/duplicates/resolve", duplicateHandler.Resolve)
|
||||||
files.POST("/bulk/tags", fileHandler.BulkSetTags)
|
files.POST("/bulk/tags", fileHandler.BulkSetTags)
|
||||||
files.POST("/bulk/delete", fileHandler.BulkDelete)
|
files.POST("/bulk/delete", fileHandler.BulkDelete)
|
||||||
|
files.POST("/bulk/review", fileHandler.BulkReview)
|
||||||
files.POST("/bulk/common-tags", fileHandler.CommonTags)
|
files.POST("/bulk/common-tags", fileHandler.CommonTags)
|
||||||
files.POST("/import", fileHandler.Import)
|
files.POST("/import", fileHandler.Import)
|
||||||
|
|
||||||
@@ -79,10 +97,12 @@ func NewRouter(
|
|||||||
files.PATCH("/:id", fileHandler.UpdateMeta)
|
files.PATCH("/:id", fileHandler.UpdateMeta)
|
||||||
files.DELETE("/:id", fileHandler.SoftDelete)
|
files.DELETE("/:id", fileHandler.SoftDelete)
|
||||||
|
|
||||||
files.GET("/:id/content", fileHandler.GetContent)
|
|
||||||
files.PUT("/:id/content", fileHandler.ReplaceContent)
|
files.PUT("/:id/content", fileHandler.ReplaceContent)
|
||||||
|
// Mints a content token (strict auth) for the GET /:id/content route below.
|
||||||
|
files.POST("/:id/content-token", fileHandler.CreateContentToken)
|
||||||
files.GET("/:id/thumbnail", fileHandler.GetThumbnail)
|
files.GET("/:id/thumbnail", fileHandler.GetThumbnail)
|
||||||
files.GET("/:id/preview", fileHandler.GetPreview)
|
files.GET("/:id/preview", fileHandler.GetPreview)
|
||||||
|
files.POST("/:id/views", fileHandler.RecordView)
|
||||||
files.POST("/:id/restore", fileHandler.Restore)
|
files.POST("/:id/restore", fileHandler.Restore)
|
||||||
files.DELETE("/:id/permanent", fileHandler.PermanentDelete)
|
files.DELETE("/:id/permanent", fileHandler.PermanentDelete)
|
||||||
|
|
||||||
@@ -93,6 +113,15 @@ func NewRouter(
|
|||||||
files.DELETE("/:id/tags/:tag_id", tagHandler.FileRemoveTag)
|
files.DELETE("/:id/tags/:tag_id", tagHandler.FileRemoveTag)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Serving an original is the one read that can outlive a 15-minute access
|
||||||
|
// token — a long video streams via repeated Range requests over many minutes.
|
||||||
|
// So this route alone also accepts a file-scoped content token (see
|
||||||
|
// HandleContent), letting the media URL stay valid for the whole playback.
|
||||||
|
media := v1.Group("/files", auth.HandleContent())
|
||||||
|
{
|
||||||
|
media.GET("/:id/content", fileHandler.GetContent)
|
||||||
|
}
|
||||||
|
|
||||||
// -------------------------------------------------------------------------
|
// -------------------------------------------------------------------------
|
||||||
// Tags (all require auth)
|
// Tags (all require auth)
|
||||||
// -------------------------------------------------------------------------
|
// -------------------------------------------------------------------------
|
||||||
@@ -139,6 +168,7 @@ func NewRouter(
|
|||||||
pools.GET("/:pool_id", poolHandler.Get)
|
pools.GET("/:pool_id", poolHandler.Get)
|
||||||
pools.PATCH("/:pool_id", poolHandler.Update)
|
pools.PATCH("/:pool_id", poolHandler.Update)
|
||||||
pools.DELETE("/:pool_id", poolHandler.Delete)
|
pools.DELETE("/:pool_id", poolHandler.Delete)
|
||||||
|
pools.POST("/:pool_id/views", poolHandler.RecordView)
|
||||||
|
|
||||||
// Sub-routes registered before /:pool_id/files to avoid param conflicts.
|
// Sub-routes registered before /:pool_id/files to avoid param conflicts.
|
||||||
pools.POST("/:pool_id/files/remove", poolHandler.RemoveFiles)
|
pools.POST("/:pool_id/files/remove", poolHandler.RemoveFiles)
|
||||||
@@ -179,5 +209,12 @@ func NewRouter(
|
|||||||
// -------------------------------------------------------------------------
|
// -------------------------------------------------------------------------
|
||||||
v1.GET("/audit", auth.Handle(), auditHandler.List)
|
v1.GET("/audit", auth.Handle(), auditHandler.List)
|
||||||
|
|
||||||
return r
|
// Serve the built single-page app on the same port as the API. When
|
||||||
}
|
// staticDir is empty (local development) the Vite dev server serves the UI
|
||||||
|
// instead, so the API runs standalone and unknown routes 404 normally.
|
||||||
|
if staticDir != "" {
|
||||||
|
r.NoRoute(spaHandler(staticDir))
|
||||||
|
}
|
||||||
|
|
||||||
|
return r, nil
|
||||||
|
}
|
||||||
|
|||||||
@@ -0,0 +1,23 @@
|
|||||||
|
package handler
|
||||||
|
|
||||||
|
import "testing"
|
||||||
|
|
||||||
|
// TestNewRouterRegisters builds the router with typed-nil dependencies to assert
|
||||||
|
// route registration itself succeeds. Gin panics on a route conflict (e.g. a
|
||||||
|
// duplicated method+path or an inconsistent wildcard name) during registration,
|
||||||
|
// before any handler runs — so this catches such mistakes without a database.
|
||||||
|
// Handlers are never invoked here; method values on nil pointers are fine.
|
||||||
|
func TestNewRouterRegisters(t *testing.T) {
|
||||||
|
r, err := NewRouter(
|
||||||
|
(*AuthMiddleware)(nil), (*AuthHandler)(nil),
|
||||||
|
(*FileHandler)(nil), (*DuplicateHandler)(nil), (*TagHandler)(nil), (*CategoryHandler)(nil), (*PoolHandler)(nil),
|
||||||
|
(*UserHandler)(nil), (*ACLHandler)(nil), (*AuditHandler)(nil),
|
||||||
|
"", nil,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("NewRouter: %v", err)
|
||||||
|
}
|
||||||
|
if r == nil {
|
||||||
|
t.Fatal("NewRouter returned nil engine")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,74 @@
|
|||||||
|
package handler
|
||||||
|
|
||||||
|
import (
|
||||||
|
"mime"
|
||||||
|
"net/http"
|
||||||
|
"os"
|
||||||
|
"path"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"github.com/gin-gonic/gin"
|
||||||
|
)
|
||||||
|
|
||||||
|
func init() {
|
||||||
|
// Go's mime table doesn't know .webmanifest; register it so the PWA manifest
|
||||||
|
// is served as JSON and isn't rejected by the X-Content-Type-Options header.
|
||||||
|
_ = mime.AddExtensionType(".webmanifest", "application/manifest+json")
|
||||||
|
}
|
||||||
|
|
||||||
|
// spaHandler serves the built single-page app from dir. It is wired as the
|
||||||
|
// router's NoRoute handler, so it only sees requests that matched no API route.
|
||||||
|
//
|
||||||
|
// A request whose path maps to a real file on disk is served directly (with
|
||||||
|
// cache headers tuned to SvelteKit's adapter-static output). Anything else
|
||||||
|
// falls back to index.html so the client-side router can resolve deep links
|
||||||
|
// like /pools/123. Unknown /api/ paths return a JSON 404 instead of the HTML
|
||||||
|
// shell, keeping API error responses machine-readable.
|
||||||
|
func spaHandler(dir string) gin.HandlerFunc {
|
||||||
|
indexPath := filepath.Join(dir, "index.html")
|
||||||
|
|
||||||
|
return func(c *gin.Context) {
|
||||||
|
reqPath := c.Request.URL.Path
|
||||||
|
|
||||||
|
if strings.HasPrefix(reqPath, "/api/") {
|
||||||
|
c.JSON(http.StatusNotFound, errorBody{
|
||||||
|
Code: "not_found",
|
||||||
|
Message: "resource not found",
|
||||||
|
})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// Resolve the request to a path inside dir. Cleaning an absolute path
|
||||||
|
// collapses any "../" segments before the join, so the result can never
|
||||||
|
// escape dir — this is the traversal guard.
|
||||||
|
clean := path.Clean("/" + reqPath)
|
||||||
|
target := filepath.Join(dir, filepath.FromSlash(clean))
|
||||||
|
|
||||||
|
if info, err := os.Stat(target); err == nil && !info.IsDir() {
|
||||||
|
c.Header("Cache-Control", cacheControl(clean))
|
||||||
|
c.File(target)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// SPA fallback: serve the shell, never cached so a new deploy is picked
|
||||||
|
// up immediately on the next navigation.
|
||||||
|
c.Header("Cache-Control", "no-cache")
|
||||||
|
c.File(indexPath)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// cacheControl returns the Cache-Control value for a served static asset.
|
||||||
|
// SvelteKit emits content-hashed files under /_app/immutable — those are safe
|
||||||
|
// to cache forever. The service worker must never be cached, or clients pin to
|
||||||
|
// a stale shell. Everything else gets a short, revalidated TTL.
|
||||||
|
func cacheControl(p string) string {
|
||||||
|
switch {
|
||||||
|
case strings.HasPrefix(p, "/_app/immutable/"):
|
||||||
|
return "public, max-age=31536000, immutable"
|
||||||
|
case p == "/service-worker.js":
|
||||||
|
return "no-cache"
|
||||||
|
default:
|
||||||
|
return "public, max-age=3600"
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -106,11 +106,11 @@ func (h *TagHandler) List(c *gin.Context) {
|
|||||||
|
|
||||||
func (h *TagHandler) Create(c *gin.Context) {
|
func (h *TagHandler) Create(c *gin.Context) {
|
||||||
var body struct {
|
var body struct {
|
||||||
Name string `json:"name" binding:"required"`
|
Name string `json:"name" binding:"required"`
|
||||||
Notes *string `json:"notes"`
|
Notes *string `json:"notes"`
|
||||||
Color *string `json:"color"`
|
Color *string `json:"color"`
|
||||||
CategoryID *string `json:"category_id"`
|
CategoryID *string `json:"category_id"`
|
||||||
IsPublic *bool `json:"is_public"`
|
IsPublic *bool `json:"is_public"`
|
||||||
}
|
}
|
||||||
if err := c.ShouldBindJSON(&body); err != nil {
|
if err := c.ShouldBindJSON(&body); err != nil {
|
||||||
respondError(c, domain.ErrValidation)
|
respondError(c, domain.ErrValidation)
|
||||||
@@ -549,4 +549,4 @@ func (h *TagHandler) FileRemoveTag(c *gin.Context) {
|
|||||||
// Helpers
|
// Helpers
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
func ptr(s string) *string { return &s }
|
func ptr(s string) *string { return &s }
|
||||||
|
|||||||
@@ -255,4 +255,4 @@ func (h *UserHandler) Delete(c *gin.Context) {
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
c.Status(http.StatusNoContent)
|
c.Status(http.StatusNoContent)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,70 @@
|
|||||||
|
// Package imagehash computes a 64-bit perceptual hash (dHash) of an image and
|
||||||
|
// compares two hashes by Hamming distance. It is used for near-duplicate
|
||||||
|
// detection: visually similar images (re-encoded, resized, recompressed) produce
|
||||||
|
// hashes a small distance apart, while unrelated images are far apart.
|
||||||
|
//
|
||||||
|
// dHash is chosen for its robustness and simplicity: the image is reduced to a
|
||||||
|
// 9×8 grayscale and each pixel is compared to its right-hand neighbour, yielding
|
||||||
|
// 64 gradient-direction bits. It tolerates scaling and brightness/contrast
|
||||||
|
// changes well, which is exactly what re-encoded duplicates exhibit.
|
||||||
|
package imagehash
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"image"
|
||||||
|
_ "image/gif" // register GIF decoder
|
||||||
|
_ "image/jpeg" // register JPEG decoder
|
||||||
|
_ "image/png" // register PNG decoder
|
||||||
|
"math/bits"
|
||||||
|
|
||||||
|
"github.com/disintegration/imaging"
|
||||||
|
_ "golang.org/x/image/webp" // register WebP decoder
|
||||||
|
)
|
||||||
|
|
||||||
|
// hashWidth/hashHeight define the reduced grayscale used for dHash. The extra
|
||||||
|
// column (width = height+1) provides the right-hand neighbour for the 64
|
||||||
|
// horizontal comparisons that make up the hash.
|
||||||
|
const (
|
||||||
|
hashHeight = 8
|
||||||
|
hashWidth = hashHeight + 1
|
||||||
|
)
|
||||||
|
|
||||||
|
// FromImage reduces img to a 9×8 grayscale and returns its 64-bit dHash. The
|
||||||
|
// uint64 of gradient bits is returned as int64 (a plain bit reinterpretation) so
|
||||||
|
// it fits PostgreSQL's bigint; equality and Distance are bitwise, so the signed
|
||||||
|
// interpretation never matters.
|
||||||
|
func FromImage(img image.Image) int64 {
|
||||||
|
small := imaging.Grayscale(imaging.Resize(img, hashWidth, hashHeight, imaging.Lanczos))
|
||||||
|
|
||||||
|
var hash uint64
|
||||||
|
bit := 0
|
||||||
|
for y := 0; y < hashHeight; y++ {
|
||||||
|
for x := 0; x < hashHeight; x++ {
|
||||||
|
// After Grayscale, R == G == B, so the red channel is the luminance.
|
||||||
|
left := small.Pix[small.PixOffset(x, y)]
|
||||||
|
right := small.Pix[small.PixOffset(x+1, y)]
|
||||||
|
if left < right {
|
||||||
|
hash |= 1 << uint(63-bit)
|
||||||
|
}
|
||||||
|
bit++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return int64(hash)
|
||||||
|
}
|
||||||
|
|
||||||
|
// FromBytes decodes data (JPEG/PNG/GIF/WebP) and returns its dHash. ok is false
|
||||||
|
// when the bytes are not a decodable image, so callers can simply skip hashing
|
||||||
|
// (e.g. leave phash NULL) rather than fail.
|
||||||
|
func FromBytes(data []byte) (hash int64, ok bool) {
|
||||||
|
img, _, err := image.Decode(bytes.NewReader(data))
|
||||||
|
if err != nil {
|
||||||
|
return 0, false
|
||||||
|
}
|
||||||
|
return FromImage(img), true
|
||||||
|
}
|
||||||
|
|
||||||
|
// Distance returns the Hamming distance (0–64) between two hashes: the number of
|
||||||
|
// differing bits. 0 means identical; small values mean near-duplicate.
|
||||||
|
func Distance(a, b int64) int {
|
||||||
|
return bits.OnesCount64(uint64(a) ^ uint64(b))
|
||||||
|
}
|
||||||
@@ -0,0 +1,99 @@
|
|||||||
|
package imagehash
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"image"
|
||||||
|
"image/color"
|
||||||
|
"image/jpeg"
|
||||||
|
"image/png"
|
||||||
|
"math"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
// radial renders a smooth grayscale image whose brightness falls off with
|
||||||
|
// distance from (cx, cy). Smooth gradients are the realistic case for perceptual
|
||||||
|
// hashing and survive JPEG re-encoding well, so they make stable test fixtures.
|
||||||
|
func radial(w, h int, cx, cy float64) image.Image {
|
||||||
|
img := image.NewRGBA(image.Rect(0, 0, w, h))
|
||||||
|
maxD := math.Hypot(float64(w), float64(h))
|
||||||
|
for y := 0; y < h; y++ {
|
||||||
|
for x := 0; x < w; x++ {
|
||||||
|
d := math.Hypot(float64(x)-cx, float64(y)-cy)
|
||||||
|
v := uint8(255 * (1 - d/maxD))
|
||||||
|
img.Set(x, y, color.RGBA{v, v, v, 255})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return img
|
||||||
|
}
|
||||||
|
|
||||||
|
func encodePNG(t *testing.T, img image.Image) []byte {
|
||||||
|
t.Helper()
|
||||||
|
var buf bytes.Buffer
|
||||||
|
if err := png.Encode(&buf, img); err != nil {
|
||||||
|
t.Fatalf("png encode: %v", err)
|
||||||
|
}
|
||||||
|
return buf.Bytes()
|
||||||
|
}
|
||||||
|
|
||||||
|
func encodeJPEG(t *testing.T, img image.Image, quality int) []byte {
|
||||||
|
t.Helper()
|
||||||
|
var buf bytes.Buffer
|
||||||
|
if err := jpeg.Encode(&buf, img, &jpeg.Options{Quality: quality}); err != nil {
|
||||||
|
t.Fatalf("jpeg encode: %v", err)
|
||||||
|
}
|
||||||
|
return buf.Bytes()
|
||||||
|
}
|
||||||
|
|
||||||
|
// The same image re-encoded as PNG (lossless) and JPEG (lossy) must hash to a
|
||||||
|
// small Hamming distance — that is the whole point of a perceptual hash.
|
||||||
|
func TestFromBytes_SameImageAcrossEncodings(t *testing.T) {
|
||||||
|
img := radial(64, 64, 32, 32)
|
||||||
|
|
||||||
|
pngHash, ok := FromBytes(encodePNG(t, img))
|
||||||
|
if !ok {
|
||||||
|
t.Fatal("FromBytes(PNG): ok=false")
|
||||||
|
}
|
||||||
|
jpgHash, ok := FromBytes(encodeJPEG(t, img, 90))
|
||||||
|
if !ok {
|
||||||
|
t.Fatal("FromBytes(JPEG): ok=false")
|
||||||
|
}
|
||||||
|
|
||||||
|
if d := Distance(pngHash, jpgHash); d > 8 {
|
||||||
|
t.Errorf("same image, different encodings: distance = %d, want <= 8", d)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Visually different images must be far apart, and clearly farther than the same
|
||||||
|
// image across encodings.
|
||||||
|
func TestDistance_DifferentImagesAreFarApart(t *testing.T) {
|
||||||
|
a := FromImage(radial(64, 64, 32, 32)) // centred
|
||||||
|
b := FromImage(radial(64, 64, 0, 0)) // corner
|
||||||
|
|
||||||
|
same, _ := FromBytes(encodeJPEG(t, radial(64, 64, 32, 32), 90))
|
||||||
|
|
||||||
|
d := Distance(a, b)
|
||||||
|
if d < 12 {
|
||||||
|
t.Errorf("different images: distance = %d, want >= 12", d)
|
||||||
|
}
|
||||||
|
if d <= Distance(a, same) {
|
||||||
|
t.Errorf("different images (%d) not farther than re-encoded same image (%d)", d, Distance(a, same))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDistance_SymmetricAndZeroForEqual(t *testing.T) {
|
||||||
|
a := FromImage(radial(64, 64, 20, 40))
|
||||||
|
b := FromImage(radial(64, 64, 40, 20))
|
||||||
|
|
||||||
|
if Distance(a, a) != 0 {
|
||||||
|
t.Errorf("Distance(a, a) = %d, want 0", Distance(a, a))
|
||||||
|
}
|
||||||
|
if Distance(a, b) != Distance(b, a) {
|
||||||
|
t.Errorf("Distance not symmetric: %d vs %d", Distance(a, b), Distance(b, a))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFromBytes_RejectsNonImage(t *testing.T) {
|
||||||
|
if _, ok := FromBytes([]byte("definitely not an image")); ok {
|
||||||
|
t.Error("FromBytes on garbage: ok=true, want false")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -22,6 +22,13 @@ type OffsetParams struct {
|
|||||||
Search string
|
Search string
|
||||||
Offset int
|
Offset int
|
||||||
Limit int
|
Limit int
|
||||||
|
|
||||||
|
// Visibility — populated by the service from the request context. When
|
||||||
|
// ViewerIsAdmin is false the repository restricts results to rows the viewer
|
||||||
|
// may see (public, owned, or explicitly granted). Ignored by user listing,
|
||||||
|
// which is admin-only.
|
||||||
|
ViewerID int16
|
||||||
|
ViewerIsAdmin bool
|
||||||
}
|
}
|
||||||
|
|
||||||
// PoolFileListParams holds parameters for listing files inside a pool.
|
// PoolFileListParams holds parameters for listing files inside a pool.
|
||||||
@@ -29,6 +36,10 @@ type PoolFileListParams struct {
|
|||||||
Cursor string
|
Cursor string
|
||||||
Limit int
|
Limit int
|
||||||
Filter string // filter DSL expression
|
Filter string // filter DSL expression
|
||||||
|
// SortKey / SortOrder come from the pool itself (not the request) and select
|
||||||
|
// how files are ordered: manual position order or an automatic file-field sort.
|
||||||
|
SortKey string
|
||||||
|
SortOrder string
|
||||||
}
|
}
|
||||||
|
|
||||||
// FileRepo is the persistence interface for file records.
|
// FileRepo is the persistence interface for file records.
|
||||||
@@ -41,6 +52,19 @@ type FileRepo interface {
|
|||||||
Create(ctx context.Context, f *domain.File) (*domain.File, error)
|
Create(ctx context.Context, f *domain.File) (*domain.File, error)
|
||||||
// Update applies partial metadata changes and returns the updated record.
|
// Update applies partial metadata changes and returns the updated record.
|
||||||
Update(ctx context.Context, id uuid.UUID, f *domain.File) (*domain.File, error)
|
Update(ctx context.Context, id uuid.UUID, f *domain.File) (*domain.File, error)
|
||||||
|
// SetNeedsReview sets the review status on the given (non-trashed) files.
|
||||||
|
SetNeedsReview(ctx context.Context, ids []uuid.UUID, value bool) error
|
||||||
|
// SetPHash sets (or clears, when nil) the perceptual hash of a file.
|
||||||
|
SetPHash(ctx context.Context, id uuid.UUID, phash *int64) error
|
||||||
|
// ListMissingPHash returns live image/video files that have no perceptual
|
||||||
|
// hash yet (the dedup backfill work list).
|
||||||
|
ListMissingPHash(ctx context.Context) ([]domain.File, error)
|
||||||
|
// ListAllPHashes returns the id and perceptual hash of every live, hashed
|
||||||
|
// file (the global input to the dedup rescan; not ACL-filtered).
|
||||||
|
ListAllPHashes(ctx context.Context) ([]domain.PHashEntry, error)
|
||||||
|
// CopyPoolMemberships adds targetID to every pool sourceID belongs to,
|
||||||
|
// skipping pools target is already in (used by the duplicate merge).
|
||||||
|
CopyPoolMemberships(ctx context.Context, targetID, sourceID uuid.UUID) error
|
||||||
// SoftDelete moves a file to trash (sets is_deleted = true).
|
// SoftDelete moves a file to trash (sets is_deleted = true).
|
||||||
SoftDelete(ctx context.Context, id uuid.UUID) error
|
SoftDelete(ctx context.Context, id uuid.UUID) error
|
||||||
// Restore moves a file out of trash (sets is_deleted = false).
|
// Restore moves a file out of trash (sets is_deleted = false).
|
||||||
@@ -52,6 +76,28 @@ type FileRepo interface {
|
|||||||
ListTags(ctx context.Context, fileID uuid.UUID) ([]domain.Tag, error)
|
ListTags(ctx context.Context, fileID uuid.UUID) ([]domain.Tag, error)
|
||||||
// SetTags replaces all tags on a file (full replace semantics).
|
// SetTags replaces all tags on a file (full replace semantics).
|
||||||
SetTags(ctx context.Context, fileID uuid.UUID, tagIDs []uuid.UUID) error
|
SetTags(ctx context.Context, fileID uuid.UUID, tagIDs []uuid.UUID) error
|
||||||
|
|
||||||
|
// RecordView appends a view-history row (activity.file_views) for the user.
|
||||||
|
RecordView(ctx context.Context, fileID uuid.UUID, userID int16) error
|
||||||
|
// RecordTagUses logs the tags referenced in a filter DSL to
|
||||||
|
// activity.tag_uses, flagging each included or excluded. Best-effort
|
||||||
|
// analytics — callers may ignore the error.
|
||||||
|
RecordTagUses(ctx context.Context, userID int16, filterDSL string) error
|
||||||
|
}
|
||||||
|
|
||||||
|
// DuplicatePairRepo persists the precomputed near-duplicate candidate pairs.
|
||||||
|
type DuplicatePairRepo interface {
|
||||||
|
// ReplaceAll atomically replaces the whole pairs table (used by the rescan).
|
||||||
|
ReplaceAll(ctx context.Context, pairs []domain.DuplicatePair) error
|
||||||
|
// ListVisible returns pairs whose both files are live, not dismissed, and
|
||||||
|
// (for non-admins) visible to the viewer.
|
||||||
|
ListVisible(ctx context.Context, viewerID int16, isAdmin bool) ([]domain.DuplicatePair, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// DismissalRepo persists "not a duplicate" decisions.
|
||||||
|
type DismissalRepo interface {
|
||||||
|
// Add records a pair as dismissed (canonical order, idempotent).
|
||||||
|
Add(ctx context.Context, a, b uuid.UUID, userID int16) error
|
||||||
}
|
}
|
||||||
|
|
||||||
// TagRepo is the persistence interface for tags.
|
// TagRepo is the persistence interface for tags.
|
||||||
@@ -87,6 +133,10 @@ type TagRuleRepo interface {
|
|||||||
// are both true, the full transitive expansion of thenTagID is retroactively
|
// are both true, the full transitive expansion of thenTagID is retroactively
|
||||||
// applied to all files that already carry whenTagID.
|
// applied to all files that already carry whenTagID.
|
||||||
SetActive(ctx context.Context, whenTagID, thenTagID uuid.UUID, active, applyToExisting bool) error
|
SetActive(ctx context.Context, whenTagID, thenTagID uuid.UUID, active, applyToExisting bool) error
|
||||||
|
// ApplyToExisting retroactively applies the full transitive expansion of
|
||||||
|
// thenTagID (following active rules) to every file that already carries
|
||||||
|
// whenTagID. Used when a rule is created or activated with apply_to_existing.
|
||||||
|
ApplyToExisting(ctx context.Context, whenTagID, thenTagID uuid.UUID) error
|
||||||
Delete(ctx context.Context, whenTagID, thenTagID uuid.UUID) error
|
Delete(ctx context.Context, whenTagID, thenTagID uuid.UUID) error
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -115,6 +165,9 @@ type PoolRepo interface {
|
|||||||
RemoveFiles(ctx context.Context, poolID uuid.UUID, fileIDs []uuid.UUID) error
|
RemoveFiles(ctx context.Context, poolID uuid.UUID, fileIDs []uuid.UUID) error
|
||||||
// Reorder sets the full ordered sequence of file IDs in the pool.
|
// Reorder sets the full ordered sequence of file IDs in the pool.
|
||||||
Reorder(ctx context.Context, poolID uuid.UUID, fileIDs []uuid.UUID) error
|
Reorder(ctx context.Context, poolID uuid.UUID, fileIDs []uuid.UUID) error
|
||||||
|
|
||||||
|
// RecordView appends a view-history row (activity.pool_views) for the user.
|
||||||
|
RecordView(ctx context.Context, poolID uuid.UUID, userID int16) error
|
||||||
}
|
}
|
||||||
|
|
||||||
// UserRepo is the persistence interface for users.
|
// UserRepo is the persistence interface for users.
|
||||||
|
|||||||
@@ -20,6 +20,11 @@ import (
|
|||||||
const (
|
const (
|
||||||
tokenTypeAccess = "access"
|
tokenTypeAccess = "access"
|
||||||
tokenTypeRefresh = "refresh"
|
tokenTypeRefresh = "refresh"
|
||||||
|
// tokenTypeContent is a file-scoped capability for reading one file's
|
||||||
|
// content by URL (originals / media streaming). It is not tied to a session,
|
||||||
|
// so it outlives the short access TTL and refresh rotation — letting a long
|
||||||
|
// video keep playing past access-token expiry.
|
||||||
|
tokenTypeContent = "content"
|
||||||
)
|
)
|
||||||
|
|
||||||
// dummyPasswordHash is a valid bcrypt hash used to equalise the cost of a login
|
// dummyPasswordHash is a valid bcrypt hash used to equalise the cost of a login
|
||||||
@@ -34,6 +39,8 @@ type Claims struct {
|
|||||||
IsAdmin bool `json:"adm"`
|
IsAdmin bool `json:"adm"`
|
||||||
SessionID int `json:"sid"`
|
SessionID int `json:"sid"`
|
||||||
TokenType string `json:"typ"`
|
TokenType string `json:"typ"`
|
||||||
|
// FileID scopes a content token to a single file; empty on access/refresh.
|
||||||
|
FileID string `json:"fid,omitempty"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// TokenPair holds an issued access/refresh token pair with the access TTL.
|
// TokenPair holds an issued access/refresh token pair with the access TTL.
|
||||||
@@ -50,6 +57,7 @@ type AuthService struct {
|
|||||||
secret []byte
|
secret []byte
|
||||||
accessTTL time.Duration
|
accessTTL time.Duration
|
||||||
refreshTTL time.Duration
|
refreshTTL time.Duration
|
||||||
|
contentTTL time.Duration
|
||||||
}
|
}
|
||||||
|
|
||||||
// NewAuthService creates an AuthService.
|
// NewAuthService creates an AuthService.
|
||||||
@@ -59,6 +67,7 @@ func NewAuthService(
|
|||||||
jwtSecret string,
|
jwtSecret string,
|
||||||
accessTTL time.Duration,
|
accessTTL time.Duration,
|
||||||
refreshTTL time.Duration,
|
refreshTTL time.Duration,
|
||||||
|
contentTTL time.Duration,
|
||||||
) *AuthService {
|
) *AuthService {
|
||||||
return &AuthService{
|
return &AuthService{
|
||||||
users: users,
|
users: users,
|
||||||
@@ -66,6 +75,7 @@ func NewAuthService(
|
|||||||
secret: []byte(jwtSecret),
|
secret: []byte(jwtSecret),
|
||||||
accessTTL: accessTTL,
|
accessTTL: accessTTL,
|
||||||
refreshTTL: refreshTTL,
|
refreshTTL: refreshTTL,
|
||||||
|
contentTTL: contentTTL,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -233,6 +243,53 @@ func (s *AuthService) ValidateAccessToken(ctx context.Context, tokenStr string)
|
|||||||
return claims, nil
|
return claims, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// GenerateContentToken issues a file-scoped capability token authorizing reads of
|
||||||
|
// one file's content (originals / media streaming) by URL. Unlike the access
|
||||||
|
// token it carries no session and is not validated against one, so it survives
|
||||||
|
// refresh rotation and outlives the short access TTL — which is what lets a long
|
||||||
|
// video keep playing. It is a bearer credential for that single file until
|
||||||
|
// ContentTokenTTL elapses. Returns the signed token and its lifetime in seconds.
|
||||||
|
func (s *AuthService) GenerateContentToken(fileID string, userID int16, isAdmin bool) (string, int, error) {
|
||||||
|
jti, err := randomJTI()
|
||||||
|
if err != nil {
|
||||||
|
return "", 0, err
|
||||||
|
}
|
||||||
|
now := time.Now()
|
||||||
|
claims := Claims{
|
||||||
|
RegisteredClaims: jwt.RegisteredClaims{
|
||||||
|
ID: jti,
|
||||||
|
IssuedAt: jwt.NewNumericDate(now),
|
||||||
|
ExpiresAt: jwt.NewNumericDate(now.Add(s.contentTTL)),
|
||||||
|
},
|
||||||
|
UserID: userID,
|
||||||
|
IsAdmin: isAdmin,
|
||||||
|
TokenType: tokenTypeContent,
|
||||||
|
FileID: fileID,
|
||||||
|
}
|
||||||
|
signed, err := s.signClaims(claims)
|
||||||
|
if err != nil {
|
||||||
|
return "", 0, err
|
||||||
|
}
|
||||||
|
return signed, int(s.contentTTL.Seconds()), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// ValidateContentToken parses a content token and checks it authorizes fileID.
|
||||||
|
// It verifies the signature and expiry (via parseToken), the content token type,
|
||||||
|
// and that the embedded file ID matches the requested file — so a token minted
|
||||||
|
// for one file cannot read another. It is intentionally session-independent (no
|
||||||
|
// session lookup), which is what lets it outlive access-token/session rotation.
|
||||||
|
// Per-file view permission is still enforced downstream against the token's user.
|
||||||
|
func (s *AuthService) ValidateContentToken(tokenStr, fileID string) (*Claims, error) {
|
||||||
|
claims, err := s.parseToken(tokenStr)
|
||||||
|
if err != nil {
|
||||||
|
return nil, domain.ErrUnauthorized
|
||||||
|
}
|
||||||
|
if claims.TokenType != tokenTypeContent || claims.FileID != fileID {
|
||||||
|
return nil, domain.ErrUnauthorized
|
||||||
|
}
|
||||||
|
return claims, nil
|
||||||
|
}
|
||||||
|
|
||||||
// issueToken signs a JWT with the given parameters. A random JWT ID guarantees
|
// issueToken signs a JWT with the given parameters. A random JWT ID guarantees
|
||||||
// uniqueness even for tokens minted within the same second.
|
// uniqueness even for tokens minted within the same second.
|
||||||
func (s *AuthService) issueToken(userID int16, isAdmin bool, sessionID int, ttl time.Duration, tokenType string) (string, error) {
|
func (s *AuthService) issueToken(userID int16, isAdmin bool, sessionID int, ttl time.Duration, tokenType string) (string, error) {
|
||||||
@@ -252,6 +309,11 @@ func (s *AuthService) issueToken(userID int16, isAdmin bool, sessionID int, ttl
|
|||||||
SessionID: sessionID,
|
SessionID: sessionID,
|
||||||
TokenType: tokenType,
|
TokenType: tokenType,
|
||||||
}
|
}
|
||||||
|
return s.signClaims(claims)
|
||||||
|
}
|
||||||
|
|
||||||
|
// signClaims signs claims into an HS256 JWT with the service secret.
|
||||||
|
func (s *AuthService) signClaims(claims Claims) (string, error) {
|
||||||
token := jwt.NewWithClaims(jwt.SigningMethodHS256, claims)
|
token := jwt.NewWithClaims(jwt.SigningMethodHS256, claims)
|
||||||
signed, err := token.SignedString(s.secret)
|
signed, err := token.SignedString(s.secret)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|||||||
@@ -0,0 +1,80 @@
|
|||||||
|
package service
|
||||||
|
|
||||||
|
import (
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// newContentTokenService builds an AuthService for content-token tests. The
|
||||||
|
// content-token methods never touch the user/session repos, so nil is fine.
|
||||||
|
func newContentTokenService(contentTTL time.Duration) *AuthService {
|
||||||
|
return NewAuthService(nil, nil, "test-secret", 15*time.Minute, 720*time.Hour, contentTTL)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestContentTokenRoundTrip(t *testing.T) {
|
||||||
|
s := newContentTokenService(time.Hour)
|
||||||
|
const fid = "11111111-1111-1111-1111-111111111111"
|
||||||
|
|
||||||
|
tok, expiresIn, err := s.GenerateContentToken(fid, 7, true)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GenerateContentToken: %v", err)
|
||||||
|
}
|
||||||
|
if expiresIn != int(time.Hour.Seconds()) {
|
||||||
|
t.Fatalf("expires_in = %d, want %d", expiresIn, int(time.Hour.Seconds()))
|
||||||
|
}
|
||||||
|
|
||||||
|
claims, err := s.ValidateContentToken(tok, fid)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ValidateContentToken: %v", err)
|
||||||
|
}
|
||||||
|
if claims.UserID != 7 || !claims.IsAdmin {
|
||||||
|
t.Fatalf("claims user mismatch: uid=%d adm=%v", claims.UserID, claims.IsAdmin)
|
||||||
|
}
|
||||||
|
if claims.FileID != fid || claims.TokenType != tokenTypeContent {
|
||||||
|
t.Fatalf("claims scope mismatch: fid=%q typ=%q", claims.FileID, claims.TokenType)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestContentTokenRejectsOtherFile(t *testing.T) {
|
||||||
|
s := newContentTokenService(time.Hour)
|
||||||
|
tok, _, err := s.GenerateContentToken("11111111-1111-1111-1111-111111111111", 7, false)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
// A token minted for one file must not authorize another.
|
||||||
|
if _, err := s.ValidateContentToken(tok, "22222222-2222-2222-2222-222222222222"); err == nil {
|
||||||
|
t.Fatal("expected rejection for a different file id")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestContentTokenRejectsAccessToken(t *testing.T) {
|
||||||
|
s := newContentTokenService(time.Hour)
|
||||||
|
// An ordinary access token must not pass as a content token (wrong type).
|
||||||
|
access, err := s.issueToken(7, false, 1, 15*time.Minute, tokenTypeAccess)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := s.ValidateContentToken(access, ""); err == nil {
|
||||||
|
t.Fatal("expected rejection of an access token as a content token")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestContentTokenRejectsExpired(t *testing.T) {
|
||||||
|
// Negative TTL → the token is already expired when minted.
|
||||||
|
s := newContentTokenService(-time.Minute)
|
||||||
|
const fid = "11111111-1111-1111-1111-111111111111"
|
||||||
|
tok, _, err := s.GenerateContentToken(fid, 7, false)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := s.ValidateContentToken(tok, fid); err == nil {
|
||||||
|
t.Fatal("expected rejection of an expired content token")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestContentTokenRejectsGarbage(t *testing.T) {
|
||||||
|
s := newContentTokenService(time.Hour)
|
||||||
|
if _, err := s.ValidateContentToken("not-a-jwt", "11111111-1111-1111-1111-111111111111"); err == nil {
|
||||||
|
t.Fatal("expected rejection of a malformed token")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -10,14 +10,14 @@ import (
|
|||||||
"tanabata/backend/internal/port"
|
"tanabata/backend/internal/port"
|
||||||
)
|
)
|
||||||
|
|
||||||
const categoryObjectType = "category"
|
const categoryObjectType = "category"
|
||||||
const categoryObjectTypeID int16 = 3 // third row in 007_seed_data.sql object_types
|
const categoryObjectTypeID int16 = 3 // third row in 007_seed_data.sql object_types
|
||||||
|
|
||||||
// CategoryParams holds the fields for creating or patching a category.
|
// CategoryParams holds the fields for creating or patching a category.
|
||||||
type CategoryParams struct {
|
type CategoryParams struct {
|
||||||
Name string
|
Name string
|
||||||
Notes *string
|
Notes *string
|
||||||
Color *string // nil = no change; pointer to empty string = clear
|
Color *string // nil = no change; pointer to empty string = clear
|
||||||
Metadata json.RawMessage
|
Metadata json.RawMessage
|
||||||
IsPublic *bool
|
IsPublic *bool
|
||||||
}
|
}
|
||||||
@@ -49,14 +49,27 @@ func NewCategoryService(
|
|||||||
// CRUD
|
// CRUD
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
// List returns a paginated, optionally filtered list of categories.
|
// List returns a paginated list of categories the caller may see.
|
||||||
func (s *CategoryService) List(ctx context.Context, params port.OffsetParams) (*domain.CategoryOffsetPage, error) {
|
func (s *CategoryService) List(ctx context.Context, params port.OffsetParams) (*domain.CategoryOffsetPage, error) {
|
||||||
|
params.ViewerID, params.ViewerIsAdmin, _ = domain.UserFromContext(ctx)
|
||||||
return s.categories.List(ctx, params)
|
return s.categories.List(ctx, params)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Get returns a category by ID.
|
// Get returns a category by ID, enforcing view ACL.
|
||||||
func (s *CategoryService) Get(ctx context.Context, id uuid.UUID) (*domain.Category, error) {
|
func (s *CategoryService) Get(ctx context.Context, id uuid.UUID) (*domain.Category, error) {
|
||||||
return s.categories.GetByID(ctx, id)
|
userID, isAdmin, _ := domain.UserFromContext(ctx)
|
||||||
|
c, err := s.categories.GetByID(ctx, id)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
ok, err := s.acl.CanView(ctx, userID, isAdmin, c.CreatorID, c.IsPublic, categoryObjectTypeID, id)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if !ok {
|
||||||
|
return nil, domain.ErrForbidden
|
||||||
|
}
|
||||||
|
return c, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// Create inserts a new category record.
|
// Create inserts a new category record.
|
||||||
@@ -158,7 +171,9 @@ func (s *CategoryService) Delete(ctx context.Context, id uuid.UUID) error {
|
|||||||
// Tags in category
|
// Tags in category
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
// ListTags returns a paginated list of tags belonging to this category.
|
// ListTags returns a paginated list of tags in this category that the caller
|
||||||
|
// may see.
|
||||||
func (s *CategoryService) ListTags(ctx context.Context, categoryID uuid.UUID, params port.OffsetParams) (*domain.TagOffsetPage, error) {
|
func (s *CategoryService) ListTags(ctx context.Context, categoryID uuid.UUID, params port.OffsetParams) (*domain.TagOffsetPage, error) {
|
||||||
|
params.ViewerID, params.ViewerIsAdmin, _ = domain.UserFromContext(ctx)
|
||||||
return s.tags.ListByCategory(ctx, categoryID, params)
|
return s.tags.ListByCategory(ctx, categoryID, params)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,173 @@
|
|||||||
|
package service
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"math/bits"
|
||||||
|
"sort"
|
||||||
|
|
||||||
|
"github.com/google/uuid"
|
||||||
|
|
||||||
|
"tanabata/backend/internal/domain"
|
||||||
|
)
|
||||||
|
|
||||||
|
// hamming returns the number of differing bits between two perceptual hashes.
|
||||||
|
func hamming(a, b uint64) int { return bits.OnesCount64(a ^ b) }
|
||||||
|
|
||||||
|
// bkNode is a node in a BK-tree over Hamming distance. Files that share the exact
|
||||||
|
// same hash are collected in ids (a distance-0 collision), so identical images
|
||||||
|
// don't degenerate the tree into a chain.
|
||||||
|
type bkNode struct {
|
||||||
|
hash uint64
|
||||||
|
ids []uuid.UUID
|
||||||
|
children map[int]*bkNode
|
||||||
|
}
|
||||||
|
|
||||||
|
// bkTree indexes perceptual hashes for sublinear radius queries. Building one and
|
||||||
|
// querying every element with a small radius is far cheaper than the O(N²) all-
|
||||||
|
// pairs comparison at 100k+ files.
|
||||||
|
type bkTree struct{ root *bkNode }
|
||||||
|
|
||||||
|
func (t *bkTree) insert(hash uint64, id uuid.UUID) {
|
||||||
|
if t.root == nil {
|
||||||
|
t.root = &bkNode{hash: hash, ids: []uuid.UUID{id}, children: map[int]*bkNode{}}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
node := t.root
|
||||||
|
for {
|
||||||
|
d := hamming(hash, node.hash)
|
||||||
|
if d == 0 {
|
||||||
|
node.ids = append(node.ids, id)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
child, ok := node.children[d]
|
||||||
|
if !ok {
|
||||||
|
node.children[d] = &bkNode{hash: hash, ids: []uuid.UUID{id}, children: map[int]*bkNode{}}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
node = child
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// query visits every node whose hash is within radius of target. The triangle
|
||||||
|
// inequality bounds which children can hold a match to [d-radius, d+radius].
|
||||||
|
func (t *bkTree) query(target uint64, radius int, visit func(node *bkNode, dist int)) {
|
||||||
|
if t.root == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
stack := []*bkNode{t.root}
|
||||||
|
for len(stack) > 0 {
|
||||||
|
node := stack[len(stack)-1]
|
||||||
|
stack = stack[:len(stack)-1]
|
||||||
|
|
||||||
|
d := hamming(target, node.hash)
|
||||||
|
if d <= radius {
|
||||||
|
visit(node, d)
|
||||||
|
}
|
||||||
|
lo, hi := d-radius, d+radius
|
||||||
|
for cd, child := range node.children {
|
||||||
|
if cd >= lo && cd <= hi {
|
||||||
|
stack = append(stack, child)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// buildPairs returns every unordered pair of files whose hashes are within
|
||||||
|
// threshold, each emitted exactly once with FileA < FileB (UUID byte order).
|
||||||
|
// onProgress, if set, is called periodically with (processed, total).
|
||||||
|
func buildPairs(entries []domain.PHashEntry, threshold int, onProgress func(done, total int)) []domain.DuplicatePair {
|
||||||
|
tree := &bkTree{}
|
||||||
|
for _, e := range entries {
|
||||||
|
tree.insert(uint64(e.PHash), e.ID)
|
||||||
|
}
|
||||||
|
|
||||||
|
var pairs []domain.DuplicatePair
|
||||||
|
total := len(entries)
|
||||||
|
for i := range entries {
|
||||||
|
e := entries[i]
|
||||||
|
tree.query(uint64(e.PHash), threshold, func(node *bkNode, dist int) {
|
||||||
|
for _, other := range node.ids {
|
||||||
|
// Emit each pair once, from the smaller id, which also skips self.
|
||||||
|
if bytes.Compare(e.ID[:], other[:]) < 0 {
|
||||||
|
pairs = append(pairs, domain.DuplicatePair{FileA: e.ID, FileB: other, Distance: dist})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
if onProgress != nil && (i+1)%1000 == 0 {
|
||||||
|
onProgress(i+1, total)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if onProgress != nil {
|
||||||
|
onProgress(total, total)
|
||||||
|
}
|
||||||
|
return pairs
|
||||||
|
}
|
||||||
|
|
||||||
|
// orderedPair returns the two ids in canonical (a < b by UUID byte order) order,
|
||||||
|
// matching how the pairs table keys a distance so a lookup hits regardless of the
|
||||||
|
// argument order.
|
||||||
|
func orderedPair(a, b uuid.UUID) [2]uuid.UUID {
|
||||||
|
if bytes.Compare(a[:], b[:]) > 0 {
|
||||||
|
return [2]uuid.UUID{b, a}
|
||||||
|
}
|
||||||
|
return [2]uuid.UUID{a, b}
|
||||||
|
}
|
||||||
|
|
||||||
|
// clusterDistances returns the stored Hamming distance for every pair of files in
|
||||||
|
// the cluster that has one. Pairs present only transitively have no stored
|
||||||
|
// distance and are left out.
|
||||||
|
func clusterDistances(files []domain.File, distByPair map[[2]uuid.UUID]int) []PairDistance {
|
||||||
|
var out []PairDistance
|
||||||
|
for i := 0; i < len(files); i++ {
|
||||||
|
for j := i + 1; j < len(files); j++ {
|
||||||
|
if d, ok := distByPair[orderedPair(files[i].ID, files[j].ID)]; ok {
|
||||||
|
out = append(out, PairDistance{A: files[i].ID, B: files[j].ID, Distance: d})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// clusterPairs groups pairs into connected components (transitive closure) via
|
||||||
|
// union-find. Every returned cluster has at least two files; clusters and the ids
|
||||||
|
// within them are sorted by UUID for stable pagination.
|
||||||
|
func clusterPairs(pairs []domain.DuplicatePair) [][]uuid.UUID {
|
||||||
|
parent := map[uuid.UUID]uuid.UUID{}
|
||||||
|
var find func(uuid.UUID) uuid.UUID
|
||||||
|
find = func(x uuid.UUID) uuid.UUID {
|
||||||
|
p, ok := parent[x]
|
||||||
|
if !ok {
|
||||||
|
parent[x] = x
|
||||||
|
return x
|
||||||
|
}
|
||||||
|
if p != x {
|
||||||
|
parent[x] = find(p)
|
||||||
|
}
|
||||||
|
return parent[x]
|
||||||
|
}
|
||||||
|
union := func(a, b uuid.UUID) {
|
||||||
|
ra, rb := find(a), find(b)
|
||||||
|
if ra != rb {
|
||||||
|
parent[ra] = rb
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, p := range pairs {
|
||||||
|
union(p.FileA, p.FileB)
|
||||||
|
}
|
||||||
|
|
||||||
|
groups := map[uuid.UUID][]uuid.UUID{}
|
||||||
|
for node := range parent {
|
||||||
|
root := find(node)
|
||||||
|
groups[root] = append(groups[root], node)
|
||||||
|
}
|
||||||
|
|
||||||
|
clusters := make([][]uuid.UUID, 0, len(groups))
|
||||||
|
for _, ids := range groups {
|
||||||
|
sort.Slice(ids, func(i, j int) bool { return bytes.Compare(ids[i][:], ids[j][:]) < 0 })
|
||||||
|
clusters = append(clusters, ids)
|
||||||
|
}
|
||||||
|
sort.Slice(clusters, func(i, j int) bool {
|
||||||
|
return bytes.Compare(clusters[i][0][:], clusters[j][0][:]) < 0
|
||||||
|
})
|
||||||
|
return clusters
|
||||||
|
}
|
||||||
@@ -0,0 +1,384 @@
|
|||||||
|
package service
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/google/uuid"
|
||||||
|
|
||||||
|
"tanabata/backend/internal/domain"
|
||||||
|
"tanabata/backend/internal/port"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Merge field source values.
|
||||||
|
const (
|
||||||
|
mergeKeep = "keep"
|
||||||
|
mergeDiscard = "discard"
|
||||||
|
mergeBoth = "both"
|
||||||
|
mergeMerge = "merge"
|
||||||
|
)
|
||||||
|
|
||||||
|
// MergeFields chooses, per field, which file supplies the survivor's value when
|
||||||
|
// resolving a duplicate. Scalars accept "keep"/"discard"; metadata also accepts
|
||||||
|
// "merge" (shallow object merge, survivor wins on key conflicts); relations
|
||||||
|
// (tags, pools) accept "keep"/"both" (union) — there is deliberately no option
|
||||||
|
// to drop the survivor's own tags/pools. An empty value defaults to "keep".
|
||||||
|
type MergeFields struct {
|
||||||
|
OriginalName string `json:"original_name"`
|
||||||
|
Notes string `json:"notes"`
|
||||||
|
ContentDatetime string `json:"content_datetime"`
|
||||||
|
IsPublic string `json:"is_public"`
|
||||||
|
Metadata string `json:"metadata"`
|
||||||
|
Tags string `json:"tags"`
|
||||||
|
Pools string `json:"pools"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// MergeSpec is the input to a duplicate resolution: keep one file, fold chosen
|
||||||
|
// fields in from the other, and (usually) trash the other.
|
||||||
|
type MergeSpec struct {
|
||||||
|
Keep uuid.UUID
|
||||||
|
Discard uuid.UUID
|
||||||
|
Fields MergeFields
|
||||||
|
DeleteDiscarded bool
|
||||||
|
}
|
||||||
|
|
||||||
|
// normalize fills empty choices with "keep" and rejects unknown values.
|
||||||
|
func (m *MergeSpec) normalize() error {
|
||||||
|
scalar := func(v *string) error {
|
||||||
|
if *v == "" {
|
||||||
|
*v = mergeKeep
|
||||||
|
}
|
||||||
|
if *v != mergeKeep && *v != mergeDiscard {
|
||||||
|
return domain.ErrValidation
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
relation := func(v *string) error {
|
||||||
|
if *v == "" {
|
||||||
|
*v = mergeKeep
|
||||||
|
}
|
||||||
|
if *v != mergeKeep && *v != mergeBoth {
|
||||||
|
return domain.ErrValidation
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
f := &m.Fields
|
||||||
|
if err := scalar(&f.OriginalName); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := scalar(&f.Notes); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := scalar(&f.ContentDatetime); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := scalar(&f.IsPublic); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if f.Metadata == "" {
|
||||||
|
f.Metadata = mergeKeep
|
||||||
|
}
|
||||||
|
if f.Metadata != mergeKeep && f.Metadata != mergeDiscard && f.Metadata != mergeMerge {
|
||||||
|
return domain.ErrValidation
|
||||||
|
}
|
||||||
|
if err := relation(&f.Tags); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := relation(&f.Pools); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// DuplicateService finds near-duplicate clusters and resolves them.
|
||||||
|
type DuplicateService struct {
|
||||||
|
files port.FileRepo
|
||||||
|
pairs port.DuplicatePairRepo
|
||||||
|
dismissals port.DismissalRepo
|
||||||
|
acl *ACLService
|
||||||
|
audit *AuditService
|
||||||
|
tx port.Transactor
|
||||||
|
threshold int
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewDuplicateService creates a DuplicateService. threshold is the maximum
|
||||||
|
// Hamming distance for two files to be treated as duplicate candidates.
|
||||||
|
func NewDuplicateService(
|
||||||
|
files port.FileRepo,
|
||||||
|
pairs port.DuplicatePairRepo,
|
||||||
|
dismissals port.DismissalRepo,
|
||||||
|
acl *ACLService,
|
||||||
|
audit *AuditService,
|
||||||
|
tx port.Transactor,
|
||||||
|
threshold int,
|
||||||
|
) *DuplicateService {
|
||||||
|
return &DuplicateService{
|
||||||
|
files: files,
|
||||||
|
pairs: pairs,
|
||||||
|
dismissals: dismissals,
|
||||||
|
acl: acl,
|
||||||
|
audit: audit,
|
||||||
|
tx: tx,
|
||||||
|
threshold: threshold,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Cluster is a group of near-duplicate files together with the pairwise Hamming
|
||||||
|
// distances known between them. Distances are read from the stored pairs, so two
|
||||||
|
// files linked into the cluster only transitively (through an intermediate) may
|
||||||
|
// have no direct distance — that pair is simply omitted.
|
||||||
|
type Cluster struct {
|
||||||
|
Files []domain.File
|
||||||
|
Distances []PairDistance
|
||||||
|
}
|
||||||
|
|
||||||
|
// PairDistance is the stored Hamming distance between two files of a cluster.
|
||||||
|
type PairDistance struct {
|
||||||
|
A uuid.UUID
|
||||||
|
B uuid.UUID
|
||||||
|
Distance int
|
||||||
|
}
|
||||||
|
|
||||||
|
// Clusters returns a page of duplicate clusters visible to the caller. Pairs are
|
||||||
|
// read from the precomputed table (no all-pairs scan here) and grouped into
|
||||||
|
// connected components; pagination is over whole clusters. Each cluster carries
|
||||||
|
// the stored pairwise distances so callers can show how close the files are.
|
||||||
|
func (s *DuplicateService) Clusters(ctx context.Context, limit, offset int) (clusters []Cluster, total int, err error) {
|
||||||
|
userID, isAdmin, _ := domain.UserFromContext(ctx)
|
||||||
|
|
||||||
|
pairs, err := s.pairs.ListVisible(ctx, userID, isAdmin)
|
||||||
|
if err != nil {
|
||||||
|
return nil, 0, err
|
||||||
|
}
|
||||||
|
groups := clusterPairs(pairs)
|
||||||
|
total = len(groups)
|
||||||
|
|
||||||
|
if offset < 0 {
|
||||||
|
offset = 0
|
||||||
|
}
|
||||||
|
if offset >= len(groups) {
|
||||||
|
return []Cluster{}, total, nil
|
||||||
|
}
|
||||||
|
end := offset + limit
|
||||||
|
if end > len(groups) || limit <= 0 {
|
||||||
|
end = len(groups)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Index the stored distances once; each page cluster looks up its own pairs.
|
||||||
|
distByPair := make(map[[2]uuid.UUID]int, len(pairs))
|
||||||
|
for _, p := range pairs {
|
||||||
|
distByPair[orderedPair(p.FileA, p.FileB)] = p.Distance
|
||||||
|
}
|
||||||
|
|
||||||
|
out := make([]Cluster, 0, end-offset)
|
||||||
|
for _, ids := range groups[offset:end] {
|
||||||
|
files := make([]domain.File, 0, len(ids))
|
||||||
|
for _, id := range ids {
|
||||||
|
f, err := s.files.GetByID(ctx, id)
|
||||||
|
if err != nil {
|
||||||
|
// A file deleted between the pair read and now just drops out.
|
||||||
|
if errors.Is(err, domain.ErrNotFound) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
return nil, 0, err
|
||||||
|
}
|
||||||
|
files = append(files, *f)
|
||||||
|
}
|
||||||
|
if len(files) >= 2 {
|
||||||
|
out = append(out, Cluster{Files: files, Distances: clusterDistances(files, distByPair)})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out, total, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Rescan recomputes the entire duplicate_pairs table from the current set of
|
||||||
|
// perceptual hashes. It is the only thing that populates the table, so the
|
||||||
|
// duplicates view reflects state as of the last rescan. Called by the dedup CLI.
|
||||||
|
func (s *DuplicateService) Rescan(ctx context.Context, onProgress func(done, total int)) error {
|
||||||
|
entries, err := s.files.ListAllPHashes(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
pairs := buildPairs(entries, s.threshold, onProgress)
|
||||||
|
return s.pairs.ReplaceAll(ctx, pairs)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Dismiss records two files as "not a duplicate" so the pair stops surfacing.
|
||||||
|
// The caller must be able to view both files.
|
||||||
|
func (s *DuplicateService) Dismiss(ctx context.Context, a, b uuid.UUID) error {
|
||||||
|
if a == b {
|
||||||
|
return domain.ErrValidation
|
||||||
|
}
|
||||||
|
userID, isAdmin, _ := domain.UserFromContext(ctx)
|
||||||
|
for _, id := range []uuid.UUID{a, b} {
|
||||||
|
f, err := s.files.GetByID(ctx, id)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
ok, err := s.acl.CanView(ctx, userID, isAdmin, f.CreatorID, f.IsPublic, fileObjectTypeID, id)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if !ok {
|
||||||
|
return domain.ErrForbidden
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := s.dismissals.Add(ctx, a, b, userID); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
objType := fileObjectType
|
||||||
|
_ = s.audit.Log(ctx, "duplicate_dismiss", &objType, &a, map[string]any{"other": b.String()})
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Resolve merges a duplicate pair: the survivor (keep) takes the chosen fields
|
||||||
|
// from the other (discard), and the other is trashed when DeleteDiscarded is set.
|
||||||
|
// The caller must be able to edit both files. Returns the updated survivor.
|
||||||
|
func (s *DuplicateService) Resolve(ctx context.Context, spec MergeSpec) (*domain.File, error) {
|
||||||
|
if spec.Keep == spec.Discard {
|
||||||
|
return nil, domain.ErrValidation
|
||||||
|
}
|
||||||
|
if err := spec.normalize(); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
keep, err := s.files.GetByID(ctx, spec.Keep)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
discard, err := s.files.GetByID(ctx, spec.Discard)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
userID, isAdmin, _ := domain.UserFromContext(ctx)
|
||||||
|
for _, f := range []*domain.File{keep, discard} {
|
||||||
|
ok, err := s.acl.CanEdit(ctx, userID, isAdmin, f.CreatorID, fileObjectTypeID, f.ID)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if !ok {
|
||||||
|
return nil, domain.ErrForbidden
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// FileRepo.Update rewrites all editable scalar columns, so build the complete
|
||||||
|
// resolved set (each field from keep or discard) rather than a sparse patch.
|
||||||
|
patch := &domain.File{
|
||||||
|
OriginalName: pickPtr(spec.Fields.OriginalName, keep.OriginalName, discard.OriginalName),
|
||||||
|
Notes: pickPtr(spec.Fields.Notes, keep.Notes, discard.Notes),
|
||||||
|
ContentDatetime: pickTime(spec.Fields.ContentDatetime, keep.ContentDatetime, discard.ContentDatetime),
|
||||||
|
IsPublic: pickBool(spec.Fields.IsPublic, keep.IsPublic, discard.IsPublic),
|
||||||
|
Metadata: pickMetadata(spec.Fields.Metadata, keep.Metadata, discard.Metadata),
|
||||||
|
}
|
||||||
|
|
||||||
|
var result *domain.File
|
||||||
|
txErr := s.tx.WithTx(ctx, func(ctx context.Context) error {
|
||||||
|
updated, err := s.files.Update(ctx, keep.ID, patch)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
if spec.Fields.Tags == mergeBoth {
|
||||||
|
if err := s.files.SetTags(ctx, keep.ID, unionTagIDs(keep.Tags, discard.Tags)); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
tags, err := s.files.ListTags(ctx, keep.ID)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
updated.Tags = tags
|
||||||
|
}
|
||||||
|
if spec.Fields.Pools == mergeBoth {
|
||||||
|
if err := s.files.CopyPoolMemberships(ctx, keep.ID, discard.ID); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if spec.DeleteDiscarded {
|
||||||
|
if err := s.files.SoftDelete(ctx, discard.ID); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
result = updated
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
if txErr != nil {
|
||||||
|
return nil, txErr
|
||||||
|
}
|
||||||
|
|
||||||
|
objType := fileObjectType
|
||||||
|
_ = s.audit.Log(ctx, "file_merge", &objType, &keep.ID, map[string]any{
|
||||||
|
"discard": spec.Discard.String(),
|
||||||
|
"fields": spec.Fields,
|
||||||
|
"deleted_discarded": spec.DeleteDiscarded,
|
||||||
|
})
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- field pickers ---------------------------------------------------------
|
||||||
|
|
||||||
|
func pickPtr(choice string, keep, discard *string) *string {
|
||||||
|
if choice == mergeDiscard {
|
||||||
|
return discard
|
||||||
|
}
|
||||||
|
return keep
|
||||||
|
}
|
||||||
|
|
||||||
|
func pickBool(choice string, keep, discard bool) bool {
|
||||||
|
if choice == mergeDiscard {
|
||||||
|
return discard
|
||||||
|
}
|
||||||
|
return keep
|
||||||
|
}
|
||||||
|
|
||||||
|
func pickTime(choice string, keep, discard time.Time) time.Time {
|
||||||
|
if choice == mergeDiscard {
|
||||||
|
return discard
|
||||||
|
}
|
||||||
|
return keep
|
||||||
|
}
|
||||||
|
|
||||||
|
func unionTagIDs(a, b []domain.Tag) []uuid.UUID {
|
||||||
|
seen := make(map[uuid.UUID]bool, len(a)+len(b))
|
||||||
|
ids := make([]uuid.UUID, 0, len(a)+len(b))
|
||||||
|
for _, t := range append(append([]domain.Tag{}, a...), b...) {
|
||||||
|
if !seen[t.ID] {
|
||||||
|
seen[t.ID] = true
|
||||||
|
ids = append(ids, t.ID)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return ids
|
||||||
|
}
|
||||||
|
|
||||||
|
// pickMetadata returns keep's metadata, discard's, or a shallow merge in which
|
||||||
|
// the survivor's keys win on conflict.
|
||||||
|
func pickMetadata(choice string, keep, discard json.RawMessage) json.RawMessage {
|
||||||
|
switch choice {
|
||||||
|
case mergeDiscard:
|
||||||
|
return discard
|
||||||
|
case mergeMerge:
|
||||||
|
km := map[string]json.RawMessage{}
|
||||||
|
dm := map[string]json.RawMessage{}
|
||||||
|
_ = json.Unmarshal(keep, &km)
|
||||||
|
_ = json.Unmarshal(discard, &dm)
|
||||||
|
out := make(map[string]json.RawMessage, len(km)+len(dm))
|
||||||
|
for k, v := range dm {
|
||||||
|
out[k] = v
|
||||||
|
}
|
||||||
|
for k, v := range km { // survivor wins
|
||||||
|
out[k] = v
|
||||||
|
}
|
||||||
|
if len(out) == 0 {
|
||||||
|
return keep
|
||||||
|
}
|
||||||
|
b, err := json.Marshal(out)
|
||||||
|
if err != nil {
|
||||||
|
return keep
|
||||||
|
}
|
||||||
|
return b
|
||||||
|
default:
|
||||||
|
return keep
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,152 @@
|
|||||||
|
package service
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"sort"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/google/uuid"
|
||||||
|
|
||||||
|
"tanabata/backend/internal/domain"
|
||||||
|
)
|
||||||
|
|
||||||
|
// id builds a deterministic UUID whose byte order matches n, so tests can reason
|
||||||
|
// about the canonical (FileA < FileB) ordering buildPairs produces.
|
||||||
|
func id(n int) uuid.UUID {
|
||||||
|
return uuid.MustParse(fmt.Sprintf("00000000-0000-0000-0000-%012d", n))
|
||||||
|
}
|
||||||
|
|
||||||
|
func entry(n int, hash uint64) domain.PHashEntry {
|
||||||
|
return domain.PHashEntry{ID: id(n), PHash: int64(hash)}
|
||||||
|
}
|
||||||
|
|
||||||
|
// pairKey canonicalises a pair for set comparison regardless of emission order.
|
||||||
|
func pairKey(p domain.DuplicatePair) string {
|
||||||
|
a, b := p.FileA, p.FileB
|
||||||
|
if bytes.Compare(a[:], b[:]) > 0 {
|
||||||
|
a, b = b, a
|
||||||
|
}
|
||||||
|
return fmt.Sprintf("%s|%s|%d", a, b, p.Distance)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestBuildPairs_ThresholdAndCanonicalOrder(t *testing.T) {
|
||||||
|
entries := []domain.PHashEntry{
|
||||||
|
entry(1, 0x0000000000000000),
|
||||||
|
entry(2, 0x0000000000000001), // distance 1 from #1
|
||||||
|
entry(3, 0x00000000000000FF), // distance 8 from #1, 7 from #2
|
||||||
|
entry(4, 0xFFFFFFFFFFFFFFFF), // distance 64 from #1
|
||||||
|
}
|
||||||
|
|
||||||
|
// Tight threshold: only the distance-1 pair qualifies.
|
||||||
|
got := buildPairs(entries, 2, nil)
|
||||||
|
if len(got) != 1 {
|
||||||
|
t.Fatalf("threshold 2: got %d pairs, want 1: %+v", len(got), got)
|
||||||
|
}
|
||||||
|
if got[0].FileA != id(1) || got[0].FileB != id(2) || got[0].Distance != 1 {
|
||||||
|
t.Errorf("threshold 2: unexpected pair %+v", got[0])
|
||||||
|
}
|
||||||
|
// Canonical order always FileA < FileB.
|
||||||
|
if bytes.Compare(got[0].FileA[:], got[0].FileB[:]) >= 0 {
|
||||||
|
t.Error("pair not in canonical FileA < FileB order")
|
||||||
|
}
|
||||||
|
|
||||||
|
// Looser threshold pulls in #3's pairs but never #4.
|
||||||
|
got8 := buildPairs(entries, 8, nil)
|
||||||
|
want := map[string]bool{
|
||||||
|
pairKey(domain.DuplicatePair{FileA: id(1), FileB: id(2), Distance: 1}): true,
|
||||||
|
pairKey(domain.DuplicatePair{FileA: id(1), FileB: id(3), Distance: 8}): true,
|
||||||
|
pairKey(domain.DuplicatePair{FileA: id(2), FileB: id(3), Distance: 7}): true,
|
||||||
|
}
|
||||||
|
if len(got8) != len(want) {
|
||||||
|
t.Fatalf("threshold 8: got %d pairs, want %d: %+v", len(got8), len(want), got8)
|
||||||
|
}
|
||||||
|
for _, p := range got8 {
|
||||||
|
if !want[pairKey(p)] {
|
||||||
|
t.Errorf("threshold 8: unexpected pair %+v", p)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestBuildPairs_IdenticalHashesPairAtDistanceZero(t *testing.T) {
|
||||||
|
entries := []domain.PHashEntry{
|
||||||
|
entry(1, 0xABCDABCDABCDABCD),
|
||||||
|
entry(2, 0xABCDABCDABCDABCD),
|
||||||
|
}
|
||||||
|
got := buildPairs(entries, 0, nil)
|
||||||
|
if len(got) != 1 || got[0].Distance != 0 || got[0].FileA != id(1) || got[0].FileB != id(2) {
|
||||||
|
t.Fatalf("identical hashes: got %+v, want one distance-0 pair (1,2)", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestClusterPairs_ConnectedComponents(t *testing.T) {
|
||||||
|
pairs := []domain.DuplicatePair{
|
||||||
|
{FileA: id(1), FileB: id(2)},
|
||||||
|
{FileA: id(2), FileB: id(3)}, // transitively joins 1-2-3
|
||||||
|
{FileA: id(5), FileB: id(6)},
|
||||||
|
}
|
||||||
|
clusters := clusterPairs(pairs)
|
||||||
|
if len(clusters) != 2 {
|
||||||
|
t.Fatalf("got %d clusters, want 2: %+v", len(clusters), clusters)
|
||||||
|
}
|
||||||
|
// Sorted by smallest id: {1,2,3} then {5,6}.
|
||||||
|
if len(clusters[0]) != 3 || clusters[0][0] != id(1) || clusters[0][2] != id(3) {
|
||||||
|
t.Errorf("cluster 0 = %v, want [1 2 3]", clusters[0])
|
||||||
|
}
|
||||||
|
if len(clusters[1]) != 2 || clusters[1][0] != id(5) {
|
||||||
|
t.Errorf("cluster 1 = %v, want [5 6]", clusters[1])
|
||||||
|
}
|
||||||
|
// Each cluster's ids are sorted.
|
||||||
|
for _, c := range clusters {
|
||||||
|
if !sort.SliceIsSorted(c, func(i, j int) bool { return bytes.Compare(c[i][:], c[j][:]) < 0 }) {
|
||||||
|
t.Errorf("cluster not sorted: %v", c)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPickMetadata_Merge(t *testing.T) {
|
||||||
|
keep := json.RawMessage(`{"a":1,"b":2}`)
|
||||||
|
discard := json.RawMessage(`{"b":9,"c":3}`)
|
||||||
|
|
||||||
|
out := pickMetadata(mergeMerge, keep, discard)
|
||||||
|
var m map[string]int
|
||||||
|
if err := json.Unmarshal(out, &m); err != nil {
|
||||||
|
t.Fatalf("merge result not valid JSON: %v (%s)", err, out)
|
||||||
|
}
|
||||||
|
want := map[string]int{"a": 1, "b": 2, "c": 3} // survivor wins on "b"
|
||||||
|
if fmt.Sprint(m) != fmt.Sprint(want) {
|
||||||
|
t.Errorf("merge = %v, want %v", m, want)
|
||||||
|
}
|
||||||
|
|
||||||
|
if string(pickMetadata(mergeKeep, keep, discard)) != string(keep) {
|
||||||
|
t.Error("keep choice should return survivor metadata unchanged")
|
||||||
|
}
|
||||||
|
if string(pickMetadata(mergeDiscard, keep, discard)) != string(discard) {
|
||||||
|
t.Error("discard choice should return the other file's metadata")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestMergeSpec_Normalize(t *testing.T) {
|
||||||
|
// Empty fields default to "keep".
|
||||||
|
spec := MergeSpec{Keep: id(1), Discard: id(2)}
|
||||||
|
if err := spec.normalize(); err != nil {
|
||||||
|
t.Fatalf("normalize empty: %v", err)
|
||||||
|
}
|
||||||
|
if spec.Fields.OriginalName != mergeKeep || spec.Fields.Tags != mergeKeep || spec.Fields.Metadata != mergeKeep {
|
||||||
|
t.Errorf("empty fields not defaulted to keep: %+v", spec.Fields)
|
||||||
|
}
|
||||||
|
|
||||||
|
// "both" is invalid for a scalar field.
|
||||||
|
bad := MergeSpec{Keep: id(1), Discard: id(2), Fields: MergeFields{Notes: mergeBoth}}
|
||||||
|
if err := bad.normalize(); !errors.Is(err, domain.ErrValidation) {
|
||||||
|
t.Errorf("scalar=both: got %v, want ErrValidation", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// "discard" is invalid for a relation field.
|
||||||
|
badRel := MergeSpec{Keep: id(1), Discard: id(2), Fields: MergeFields{Tags: mergeDiscard}}
|
||||||
|
if err := badRel.normalize(); !errors.Is(err, domain.ErrValidation) {
|
||||||
|
t.Errorf("relation=discard: got %v, want ErrValidation", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -8,14 +8,15 @@ import (
|
|||||||
"io"
|
"io"
|
||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
|
"sort"
|
||||||
"strings"
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/gabriel-vasile/mimetype"
|
"github.com/gabriel-vasile/mimetype"
|
||||||
"github.com/google/uuid"
|
"github.com/google/uuid"
|
||||||
"github.com/rwcarlsen/goexif/exif"
|
|
||||||
|
|
||||||
"tanabata/backend/internal/domain"
|
"tanabata/backend/internal/domain"
|
||||||
|
"tanabata/backend/internal/imagehash"
|
||||||
"tanabata/backend/internal/port"
|
"tanabata/backend/internal/port"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -33,8 +34,12 @@ type UploadParams struct {
|
|||||||
Notes *string
|
Notes *string
|
||||||
Metadata json.RawMessage
|
Metadata json.RawMessage
|
||||||
ContentDatetime *time.Time
|
ContentDatetime *time.Time
|
||||||
IsPublic bool
|
// ContentDatetimeFallback is used for content_datetime only when neither an
|
||||||
TagIDs []uuid.UUID
|
// explicit ContentDatetime nor an EXIF date is available (e.g. the source
|
||||||
|
// file's mtime on a server-side import).
|
||||||
|
ContentDatetimeFallback *time.Time
|
||||||
|
IsPublic bool
|
||||||
|
TagIDs []uuid.UUID
|
||||||
}
|
}
|
||||||
|
|
||||||
// UpdateParams holds the parameters for updating file metadata.
|
// UpdateParams holds the parameters for updating file metadata.
|
||||||
@@ -67,6 +72,24 @@ type ImportResult struct {
|
|||||||
Errors []ImportFileError `json:"errors"`
|
Errors []ImportFileError `json:"errors"`
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ImportEvent is one progress message streamed during an import, letting the UI
|
||||||
|
// show a live progress bar and a per-file status list. Type is the discriminator:
|
||||||
|
//
|
||||||
|
// "start" — total is the number of entries about to be processed.
|
||||||
|
// "file" — one entry finished: index (1-based), filename, status, optional reason.
|
||||||
|
// "done" — final tallies (imported/skipped/errors).
|
||||||
|
type ImportEvent struct {
|
||||||
|
Type string `json:"type"`
|
||||||
|
Total int `json:"total,omitempty"`
|
||||||
|
Index int `json:"index,omitempty"`
|
||||||
|
Filename string `json:"filename,omitempty"`
|
||||||
|
Status string `json:"status,omitempty"` // "imported" | "skipped" | "error"
|
||||||
|
Reason string `json:"reason,omitempty"`
|
||||||
|
Imported int `json:"imported,omitempty"`
|
||||||
|
Skipped int `json:"skipped,omitempty"`
|
||||||
|
Errors int `json:"errors,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
// FileService handles business logic for file records.
|
// FileService handles business logic for file records.
|
||||||
type FileService struct {
|
type FileService struct {
|
||||||
files port.FileRepo
|
files port.FileRepo
|
||||||
@@ -108,7 +131,7 @@ func NewFileService(
|
|||||||
|
|
||||||
// Upload validates the MIME type, saves the file to storage, creates the DB
|
// Upload validates the MIME type, saves the file to storage, creates the DB
|
||||||
// record, and applies any initial tags — all within a single transaction.
|
// record, and applies any initial tags — all within a single transaction.
|
||||||
// If ContentDatetime is nil and EXIF DateTimeOriginal is present, it is used.
|
// If ContentDatetime is nil and the metadata carries a capture date, it is used.
|
||||||
func (s *FileService) Upload(ctx context.Context, p UploadParams) (*domain.File, error) {
|
func (s *FileService) Upload(ctx context.Context, p UploadParams) (*domain.File, error) {
|
||||||
userID, _, _ := domain.UserFromContext(ctx)
|
userID, _, _ := domain.UserFromContext(ctx)
|
||||||
|
|
||||||
@@ -125,15 +148,32 @@ func (s *FileService) Upload(ctx context.Context, p UploadParams) (*domain.File,
|
|||||||
}
|
}
|
||||||
data := buf.Bytes()
|
data := buf.Bytes()
|
||||||
|
|
||||||
// Extract EXIF metadata (best-effort; non-image files will error silently).
|
// Extract rich metadata (best-effort; covers images, video and audio).
|
||||||
exifData, exifDatetime := extractEXIFWithDatetime(data)
|
var origName string
|
||||||
|
if p.OriginalName != nil {
|
||||||
|
origName = *p.OriginalName
|
||||||
|
}
|
||||||
|
exifData, exifDatetime := extractMetadata(data, origName, p.ContentDatetimeFallback)
|
||||||
|
|
||||||
// Resolve content datetime: explicit > EXIF > zero value.
|
// Compute a perceptual hash for images so duplicate detection can later match
|
||||||
|
// near-identical files. Best-effort: a decode failure just leaves phash unset
|
||||||
|
// (the dedup CLI backfills it). Video is hashed by that CLI, not inline, to keep
|
||||||
|
// ffmpeg off the upload path.
|
||||||
|
var phash *int64
|
||||||
|
if strings.HasPrefix(mime.Name, "image/") {
|
||||||
|
if h, ok := imagehash.FromBytes(data); ok {
|
||||||
|
phash = &h
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Resolve content datetime: explicit > metadata date > fallback (e.g. import mtime) > zero.
|
||||||
var contentDatetime time.Time
|
var contentDatetime time.Time
|
||||||
if p.ContentDatetime != nil {
|
if p.ContentDatetime != nil {
|
||||||
contentDatetime = *p.ContentDatetime
|
contentDatetime = *p.ContentDatetime
|
||||||
} else if exifDatetime != nil {
|
} else if exifDatetime != nil {
|
||||||
contentDatetime = *exifDatetime
|
contentDatetime = *exifDatetime
|
||||||
|
} else if p.ContentDatetimeFallback != nil {
|
||||||
|
contentDatetime = *p.ContentDatetimeFallback
|
||||||
}
|
}
|
||||||
|
|
||||||
// Assign UUID v7 so CreatedAt can be derived from it later.
|
// Assign UUID v7 so CreatedAt can be derived from it later.
|
||||||
@@ -159,6 +199,7 @@ func (s *FileService) Upload(ctx context.Context, p UploadParams) (*domain.File,
|
|||||||
Notes: p.Notes,
|
Notes: p.Notes,
|
||||||
Metadata: p.Metadata,
|
Metadata: p.Metadata,
|
||||||
EXIF: exifData,
|
EXIF: exifData,
|
||||||
|
PHash: phash,
|
||||||
CreatorID: userID,
|
CreatorID: userID,
|
||||||
IsPublic: p.IsPublic,
|
IsPublic: p.IsPublic,
|
||||||
}
|
}
|
||||||
@@ -208,6 +249,26 @@ func (s *FileService) Get(ctx context.Context, id uuid.UUID) (*domain.File, erro
|
|||||||
return f, nil
|
return f, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// RecordView appends a view-history entry for the current user, enforcing view
|
||||||
|
// ACL (you can only record a view of a file you may see).
|
||||||
|
func (s *FileService) RecordView(ctx context.Context, id uuid.UUID) error {
|
||||||
|
userID, isAdmin, _ := domain.UserFromContext(ctx)
|
||||||
|
|
||||||
|
f, err := s.files.GetByID(ctx, id)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
ok, err := s.acl.CanView(ctx, userID, isAdmin, f.CreatorID, f.IsPublic, fileObjectTypeID, id)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if !ok {
|
||||||
|
return domain.ErrForbidden
|
||||||
|
}
|
||||||
|
return s.files.RecordView(ctx, id, userID)
|
||||||
|
}
|
||||||
|
|
||||||
// Update applies metadata changes to a file, enforcing edit ACL.
|
// Update applies metadata changes to a file, enforcing edit ACL.
|
||||||
func (s *FileService) Update(ctx context.Context, id uuid.UUID, p UpdateParams) (*domain.File, error) {
|
func (s *FileService) Update(ctx context.Context, id uuid.UUID, p UpdateParams) (*domain.File, error) {
|
||||||
userID, isAdmin, _ := domain.UserFromContext(ctx)
|
userID, isAdmin, _ := domain.UserFromContext(ctx)
|
||||||
@@ -225,7 +286,17 @@ func (s *FileService) Update(ctx context.Context, id uuid.UUID, p UpdateParams)
|
|||||||
return nil, domain.ErrForbidden
|
return nil, domain.ErrForbidden
|
||||||
}
|
}
|
||||||
|
|
||||||
patch := &domain.File{}
|
// The repo rewrites every editable column, so the patch must carry the final
|
||||||
|
// value for each. Seed the fields that have no always-present input in the
|
||||||
|
// editor (original_name, metadata) with their current values so a partial
|
||||||
|
// update that omits them leaves them untouched instead of clearing them. The
|
||||||
|
// remaining scalars are always supplied by the editor (notes uses an explicit
|
||||||
|
// null to clear). The merge path builds its own complete patch and calls the
|
||||||
|
// repo directly, so it is unaffected by this.
|
||||||
|
patch := &domain.File{
|
||||||
|
OriginalName: f.OriginalName,
|
||||||
|
Metadata: f.Metadata,
|
||||||
|
}
|
||||||
if p.OriginalName != nil {
|
if p.OriginalName != nil {
|
||||||
patch.OriginalName = p.OriginalName
|
patch.OriginalName = p.OriginalName
|
||||||
}
|
}
|
||||||
@@ -379,7 +450,11 @@ func (s *FileService) Replace(ctx context.Context, id uuid.UUID, p UploadParams)
|
|||||||
return nil, fmt.Errorf("FileService.Replace: read body: %w", err)
|
return nil, fmt.Errorf("FileService.Replace: read body: %w", err)
|
||||||
}
|
}
|
||||||
data := buf.Bytes()
|
data := buf.Bytes()
|
||||||
exifData, _ := extractEXIFWithDatetime(data)
|
var origName string
|
||||||
|
if p.OriginalName != nil {
|
||||||
|
origName = *p.OriginalName
|
||||||
|
}
|
||||||
|
exifData, _ := extractMetadata(data, origName, nil)
|
||||||
|
|
||||||
if _, err := s.storage.Save(ctx, id, bytes.NewReader(data)); err != nil {
|
if _, err := s.storage.Save(ctx, id, bytes.NewReader(data)); err != nil {
|
||||||
return nil, fmt.Errorf("FileService.Replace: save to storage: %w", err)
|
return nil, fmt.Errorf("FileService.Replace: save to storage: %w", err)
|
||||||
@@ -401,14 +476,41 @@ func (s *FileService) Replace(ctx context.Context, id uuid.UUID, p UploadParams)
|
|||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Recompute the perceptual hash from the new content: images inline, anything
|
||||||
|
// else cleared to NULL so the old content's hash never lingers (the dedup CLI
|
||||||
|
// recomputes video). Best-effort, like on upload — phash is recomputable.
|
||||||
|
var phash *int64
|
||||||
|
if strings.HasPrefix(mime.Name, "image/") {
|
||||||
|
if h, ok := imagehash.FromBytes(data); ok {
|
||||||
|
phash = &h
|
||||||
|
}
|
||||||
|
}
|
||||||
|
_ = s.files.SetPHash(ctx, id, phash)
|
||||||
|
updated.PHash = phash
|
||||||
|
|
||||||
objType := fileObjectType
|
objType := fileObjectType
|
||||||
_ = s.audit.Log(ctx, "file_replace", &objType, &id, nil)
|
_ = s.audit.Log(ctx, "file_replace", &objType, &id, nil)
|
||||||
return updated, nil
|
return updated, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// List delegates to FileRepo with the given params.
|
// List delegates to FileRepo with the given params, restricting results to
|
||||||
|
// files the caller may see (unless they are an admin).
|
||||||
func (s *FileService) List(ctx context.Context, params domain.FileListParams) (*domain.FilePage, error) {
|
func (s *FileService) List(ctx context.Context, params domain.FileListParams) (*domain.FilePage, error) {
|
||||||
return s.files.List(ctx, params)
|
params.ViewerID, params.ViewerIsAdmin, _ = domain.UserFromContext(ctx)
|
||||||
|
|
||||||
|
page, err := s.files.List(ctx, params)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
// Log tag usage when a filter is first applied — not on pagination (cursor)
|
||||||
|
// or an anchored return, so a single browse counts once. Best-effort
|
||||||
|
// analytics; a failed write never breaks the listing.
|
||||||
|
if params.Filter != "" && params.Cursor == "" && params.Anchor == nil && params.ViewerID != 0 {
|
||||||
|
_ = s.files.RecordTagUses(ctx, params.ViewerID, params.Filter)
|
||||||
|
}
|
||||||
|
|
||||||
|
return page, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// AuthorizeView ensures the caller may view the file. Returns ErrNotFound if the
|
// AuthorizeView ensures the caller may view the file. Returns ErrNotFound if the
|
||||||
@@ -491,13 +593,59 @@ func (s *FileService) BulkDelete(ctx context.Context, fileIDs []uuid.UUID) error
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// SetNeedsReview sets the review status ("needs tagging" vs marked done) on the
|
||||||
|
// given files. Each file is checked against edit ACL; files the caller cannot
|
||||||
|
// edit, or that do not exist, are skipped (same forgiving semantics as
|
||||||
|
// BulkDelete). Authorized files are updated in a single statement and each is
|
||||||
|
// audit-logged. Works for one file (single-element slice) or many.
|
||||||
|
func (s *FileService) SetNeedsReview(ctx context.Context, ids []uuid.UUID, value bool) error {
|
||||||
|
userID, isAdmin, _ := domain.UserFromContext(ctx)
|
||||||
|
|
||||||
|
authorized := make([]uuid.UUID, 0, len(ids))
|
||||||
|
for _, id := range ids {
|
||||||
|
f, err := s.files.GetByID(ctx, id)
|
||||||
|
if err != nil {
|
||||||
|
if err == domain.ErrNotFound {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
ok, err := s.acl.CanEdit(ctx, userID, isAdmin, f.CreatorID, fileObjectTypeID, id)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if ok {
|
||||||
|
authorized = append(authorized, id)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(authorized) == 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if err := s.files.SetNeedsReview(ctx, authorized, value); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
objType := fileObjectType
|
||||||
|
details := map[string]any{"needs_review": value}
|
||||||
|
for i := range authorized {
|
||||||
|
_ = s.audit.Log(ctx, "file_review", &objType, &authorized[i], details)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// Import
|
// Import
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
// Import scans a server-side directory and uploads all supported files.
|
// Import scans a server-side directory and uploads all supported files.
|
||||||
// If path is empty, the configured default import path is used.
|
// If path is empty, the configured default import path is used.
|
||||||
func (s *FileService) Import(ctx context.Context, path string) (*ImportResult, error) {
|
//
|
||||||
|
// onProgress, when non-nil, receives a "start" event, one "file" event per
|
||||||
|
// directory entry as it is processed, and a final "done" event — letting a
|
||||||
|
// caller stream live progress. It is always called from this goroutine (never
|
||||||
|
// concurrently). The aggregate result is also returned for non-streaming callers.
|
||||||
|
func (s *FileService) Import(ctx context.Context, path string, onProgress func(ImportEvent)) (*ImportResult, error) {
|
||||||
if s.importPath == "" {
|
if s.importPath == "" {
|
||||||
return nil, domain.ErrValidation
|
return nil, domain.ErrValidation
|
||||||
}
|
}
|
||||||
@@ -518,68 +666,117 @@ func (s *FileService) Import(ctx context.Context, path string) (*ImportResult, e
|
|||||||
return nil, fmt.Errorf("FileService.Import: read dir %q: %w", dir, err)
|
return nil, fmt.Errorf("FileService.Import: read dir %q: %w", dir, err)
|
||||||
}
|
}
|
||||||
|
|
||||||
result := &ImportResult{Errors: []ImportFileError{}}
|
// Import oldest-first: process entries in ascending mtime order so the
|
||||||
|
// resulting records' creation order matches the files' chronological order.
|
||||||
|
// mtimes are cached once (re-stat'ing per comparison would be wasteful) and
|
||||||
|
// reused below as the content_datetime fallback. Entries whose info can't be
|
||||||
|
// read get a zero time (sort first); they surface their error in the loop.
|
||||||
|
modTimes := make(map[string]time.Time, len(entries))
|
||||||
|
for _, e := range entries {
|
||||||
|
if info, infoErr := e.Info(); infoErr == nil {
|
||||||
|
modTimes[e.Name()] = info.ModTime()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// SliceStable preserves ReadDir's name order as a deterministic tiebreak for
|
||||||
|
// entries sharing an mtime.
|
||||||
|
sort.SliceStable(entries, func(a, b int) bool {
|
||||||
|
return modTimes[entries[a].Name()].Before(modTimes[entries[b].Name()])
|
||||||
|
})
|
||||||
|
|
||||||
|
emit := func(ev ImportEvent) {
|
||||||
|
if onProgress != nil {
|
||||||
|
onProgress(ev)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
result := &ImportResult{Errors: []ImportFileError{}}
|
||||||
|
total := len(entries)
|
||||||
|
emit(ImportEvent{Type: "start", Total: total})
|
||||||
|
|
||||||
|
for i, entry := range entries {
|
||||||
|
name := entry.Name()
|
||||||
|
file := func(status, reason string) {
|
||||||
|
emit(ImportEvent{
|
||||||
|
Type: "file", Index: i + 1, Total: total,
|
||||||
|
Filename: name, Status: status, Reason: reason,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
fail := func(reason string) {
|
||||||
|
result.Errors = append(result.Errors, ImportFileError{Filename: name, Reason: reason})
|
||||||
|
file("error", reason)
|
||||||
|
}
|
||||||
|
|
||||||
for _, entry := range entries {
|
|
||||||
if entry.IsDir() {
|
if entry.IsDir() {
|
||||||
result.Skipped++
|
result.Skipped++
|
||||||
|
file("skipped", "directory")
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
|
|
||||||
fullPath := filepath.Join(dir, entry.Name())
|
fullPath := filepath.Join(dir, name)
|
||||||
|
|
||||||
mt, err := mimetype.DetectFile(fullPath)
|
mt, err := mimetype.DetectFile(fullPath)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
result.Errors = append(result.Errors, ImportFileError{
|
fail(fmt.Sprintf("MIME detection failed: %s", err))
|
||||||
Filename: entry.Name(),
|
|
||||||
Reason: fmt.Sprintf("MIME detection failed: %s", err),
|
|
||||||
})
|
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
|
|
||||||
mimeStr := mt.String()
|
mimeStr := mt.String()
|
||||||
// Strip parameters (e.g. "text/plain; charset=utf-8" → "text/plain").
|
// Strip parameters (e.g. "text/plain; charset=utf-8" → "text/plain").
|
||||||
if idx := len(mimeStr); idx > 0 {
|
if j := strings.IndexByte(mimeStr, ';'); j >= 0 {
|
||||||
for i, c := range mimeStr {
|
mimeStr = mimeStr[:j]
|
||||||
if c == ';' {
|
|
||||||
mimeStr = mimeStr[:i]
|
|
||||||
break
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
if _, err := s.mimes.GetByName(ctx, mimeStr); err != nil {
|
if _, err := s.mimes.GetByName(ctx, mimeStr); err != nil {
|
||||||
result.Skipped++
|
result.Skipped++
|
||||||
|
file("skipped", "unsupported type: "+mimeStr)
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
|
|
||||||
f, err := os.Open(fullPath)
|
f, err := os.Open(fullPath)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
result.Errors = append(result.Errors, ImportFileError{
|
fail(fmt.Sprintf("open failed: %s", err))
|
||||||
Filename: entry.Name(),
|
|
||||||
Reason: fmt.Sprintf("open failed: %s", err),
|
|
||||||
})
|
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
|
|
||||||
name := entry.Name()
|
// Preserve the file's mtime as a content_datetime fallback (used only when
|
||||||
|
// the file has no EXIF date) — once the source is removed below it's the
|
||||||
|
// only date left for non-photo files. Reuses the mtime cached for sorting.
|
||||||
|
var mtime *time.Time
|
||||||
|
if t, ok := modTimes[name]; ok {
|
||||||
|
mtime = &t
|
||||||
|
}
|
||||||
|
|
||||||
_, uploadErr := s.Upload(ctx, UploadParams{
|
_, uploadErr := s.Upload(ctx, UploadParams{
|
||||||
Reader: f,
|
Reader: f,
|
||||||
MIMEType: mimeStr,
|
MIMEType: mimeStr,
|
||||||
OriginalName: &name,
|
OriginalName: &name,
|
||||||
|
ContentDatetimeFallback: mtime,
|
||||||
})
|
})
|
||||||
f.Close()
|
f.Close()
|
||||||
|
|
||||||
if uploadErr != nil {
|
if uploadErr != nil {
|
||||||
result.Errors = append(result.Errors, ImportFileError{
|
fail(uploadErr.Error())
|
||||||
Filename: entry.Name(),
|
|
||||||
Reason: uploadErr.Error(),
|
|
||||||
})
|
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
result.Imported++
|
result.Imported++
|
||||||
|
|
||||||
|
// Remove the source on success so the import folder drains and re-running
|
||||||
|
// doesn't duplicate. The file is already safely copied into storage; a
|
||||||
|
// removal failure is reported but doesn't undo the import.
|
||||||
|
if rmErr := os.Remove(fullPath); rmErr != nil {
|
||||||
|
reason := fmt.Sprintf("imported, but failed to remove source: %s", rmErr)
|
||||||
|
result.Errors = append(result.Errors, ImportFileError{Filename: name, Reason: reason})
|
||||||
|
file("imported", reason) // imported, with a warning
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
file("imported", "")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
emit(ImportEvent{
|
||||||
|
Type: "done", Total: total,
|
||||||
|
Imported: result.Imported, Skipped: result.Skipped, Errors: len(result.Errors),
|
||||||
|
})
|
||||||
|
|
||||||
return result, nil
|
return result, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -608,21 +805,3 @@ func confineToBase(base, target string) (string, error) {
|
|||||||
}
|
}
|
||||||
return absTarget, nil
|
return absTarget, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// extractEXIFWithDatetime parses EXIF from raw bytes, returning both the JSON
|
|
||||||
// representation and the DateTimeOriginal (if present). Both may be nil.
|
|
||||||
func extractEXIFWithDatetime(data []byte) (json.RawMessage, *time.Time) {
|
|
||||||
x, err := exif.Decode(bytes.NewReader(data))
|
|
||||||
if err != nil {
|
|
||||||
return nil, nil
|
|
||||||
}
|
|
||||||
b, err := x.MarshalJSON()
|
|
||||||
if err != nil {
|
|
||||||
return nil, nil
|
|
||||||
}
|
|
||||||
var dt *time.Time
|
|
||||||
if t, err := x.DateTime(); err == nil {
|
|
||||||
dt = &t
|
|
||||||
}
|
|
||||||
return json.RawMessage(b), dt
|
|
||||||
}
|
|
||||||
@@ -0,0 +1,183 @@
|
|||||||
|
package service
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"os"
|
||||||
|
"os/exec"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/rwcarlsen/goexif/exif"
|
||||||
|
)
|
||||||
|
|
||||||
|
// exiftoolPath is resolved once at startup. When exiftool isn't installed we
|
||||||
|
// skip the subprocess and fall back to the pure-Go EXIF reader, so the server
|
||||||
|
// still runs (with thinner metadata) on hosts without it.
|
||||||
|
var exiftoolPath, _ = exec.LookPath("exiftool")
|
||||||
|
|
||||||
|
// metadataTimeout bounds a single exiftool invocation so a pathological file
|
||||||
|
// can't wedge an upload.
|
||||||
|
const metadataTimeout = 30 * time.Second
|
||||||
|
|
||||||
|
// metaTempFileKeys are exiftool fields that describe the temporary file we feed
|
||||||
|
// it rather than the content. Dropping them avoids leaking internal paths and
|
||||||
|
// recording the temp file's permissions/inode timestamps.
|
||||||
|
var metaTempFileKeys = []string{
|
||||||
|
"SourceFile",
|
||||||
|
"Directory",
|
||||||
|
"FileAccessDate",
|
||||||
|
"FileInodeChangeDate",
|
||||||
|
"FilePermissions",
|
||||||
|
}
|
||||||
|
|
||||||
|
// metaDateKeys are the metadata fields, in priority order, holding the moment
|
||||||
|
// the content was actually captured/created — photos first, then video atoms.
|
||||||
|
var metaDateKeys = []string{
|
||||||
|
"DateTimeOriginal",
|
||||||
|
"CreateDate",
|
||||||
|
"MediaCreateDate",
|
||||||
|
"TrackCreateDate",
|
||||||
|
"ModifyDate",
|
||||||
|
}
|
||||||
|
|
||||||
|
// extractMetadata returns rich metadata as JSON plus the best content datetime
|
||||||
|
// it can find. It prefers exiftool, which understands video, audio and every
|
||||||
|
// image format and emits machine-readable numeric values (the basis for later
|
||||||
|
// analytics); when exiftool is unavailable it falls back to the pure-Go EXIF
|
||||||
|
// reader, which only handles JPEG/TIFF.
|
||||||
|
//
|
||||||
|
// originalName supplies the extension exiftool uses for format detection and the
|
||||||
|
// FileName reported back. mtime, when set (e.g. a server-side import), is stamped
|
||||||
|
// onto the temp file so FileModifyDate reflects the real source.
|
||||||
|
func extractMetadata(data []byte, originalName string, mtime *time.Time) (json.RawMessage, *time.Time) {
|
||||||
|
if exiftoolPath != "" {
|
||||||
|
if raw, dt, ok := exiftoolExtract(data, originalName, mtime); ok {
|
||||||
|
return raw, dt
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return extractEXIFWithDatetime(data)
|
||||||
|
}
|
||||||
|
|
||||||
|
// exiftoolExtract stages the bytes in a temp file and shells out to exiftool.
|
||||||
|
// It returns ok=false on any failure so the caller can fall back.
|
||||||
|
func exiftoolExtract(data []byte, originalName string, mtime *time.Time) (json.RawMessage, *time.Time, bool) {
|
||||||
|
// exiftool reads a real file far more reliably than a pipe (it seeks freely,
|
||||||
|
// e.g. to a trailing MP4 moov atom), so stage the bytes in a temp file whose
|
||||||
|
// extension matches the original for accurate format detection.
|
||||||
|
tmp, err := os.CreateTemp("", "tfm-meta-*"+filepath.Ext(originalName))
|
||||||
|
if err != nil {
|
||||||
|
return nil, nil, false
|
||||||
|
}
|
||||||
|
tmpName := tmp.Name()
|
||||||
|
defer os.Remove(tmpName)
|
||||||
|
|
||||||
|
if _, err := tmp.Write(data); err != nil {
|
||||||
|
tmp.Close()
|
||||||
|
return nil, nil, false
|
||||||
|
}
|
||||||
|
if err := tmp.Close(); err != nil {
|
||||||
|
return nil, nil, false
|
||||||
|
}
|
||||||
|
if mtime != nil {
|
||||||
|
_ = os.Chtimes(tmpName, *mtime, *mtime)
|
||||||
|
}
|
||||||
|
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), metadataTimeout)
|
||||||
|
defer cancel()
|
||||||
|
// -n forces raw numeric/machine values for every tag (no "3.53 Mbps" strings)
|
||||||
|
// so the metadata is analytics-ready. -all extracts every tag. largefilesupport
|
||||||
|
// handles multi-GB videos. Output is a one-element JSON array.
|
||||||
|
out, err := exec.CommandContext(ctx, exiftoolPath,
|
||||||
|
"-n", "-all", "-json", "-api", "largefilesupport=1", tmpName,
|
||||||
|
).Output()
|
||||||
|
if err != nil {
|
||||||
|
return nil, nil, false
|
||||||
|
}
|
||||||
|
|
||||||
|
var arr []map[string]json.RawMessage
|
||||||
|
if err := json.Unmarshal(out, &arr); err != nil || len(arr) == 0 {
|
||||||
|
return nil, nil, false
|
||||||
|
}
|
||||||
|
m := arr[0]
|
||||||
|
|
||||||
|
dt := pickMetaDatetime(m)
|
||||||
|
|
||||||
|
// Strip temp-file artifacts and substitute the real name.
|
||||||
|
for _, k := range metaTempFileKeys {
|
||||||
|
delete(m, k)
|
||||||
|
}
|
||||||
|
if mtime == nil {
|
||||||
|
// Without a real source mtime this is just the temp file's write time.
|
||||||
|
delete(m, "FileModifyDate")
|
||||||
|
}
|
||||||
|
if originalName != "" {
|
||||||
|
if nb, err := json.Marshal(originalName); err == nil {
|
||||||
|
m["FileName"] = nb
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
delete(m, "FileName")
|
||||||
|
}
|
||||||
|
|
||||||
|
raw, err := json.Marshal(m)
|
||||||
|
if err != nil {
|
||||||
|
return nil, nil, false
|
||||||
|
}
|
||||||
|
return raw, dt, true
|
||||||
|
}
|
||||||
|
|
||||||
|
// pickMetaDatetime returns the first parseable content date among metaDateKeys.
|
||||||
|
func pickMetaDatetime(m map[string]json.RawMessage) *time.Time {
|
||||||
|
for _, key := range metaDateKeys {
|
||||||
|
raw, ok := m[key]
|
||||||
|
if !ok {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
var s string
|
||||||
|
if err := json.Unmarshal(raw, &s); err != nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if t, ok := parseExifDate(s); ok {
|
||||||
|
return &t
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// parseExifDate parses exiftool's "YYYY:MM:DD HH:MM:SS" timestamps, with or
|
||||||
|
// without a trailing timezone offset. Zeroed placeholders ("0000:00:00 ...")
|
||||||
|
// fail to parse and are skipped by the caller.
|
||||||
|
func parseExifDate(s string) (time.Time, bool) {
|
||||||
|
s = strings.TrimSpace(s)
|
||||||
|
for _, layout := range []string{
|
||||||
|
"2006:01:02 15:04:05-07:00",
|
||||||
|
"2006:01:02 15:04:05Z07:00",
|
||||||
|
"2006:01:02 15:04:05",
|
||||||
|
} {
|
||||||
|
if t, err := time.Parse(layout, s); err == nil {
|
||||||
|
return t, true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return time.Time{}, false
|
||||||
|
}
|
||||||
|
|
||||||
|
// extractEXIFWithDatetime is the pure-Go fallback used when exiftool is absent.
|
||||||
|
// It parses EXIF from raw bytes (JPEG/TIFF only), returning both the JSON
|
||||||
|
// representation and the DateTimeOriginal (if present). Both may be nil.
|
||||||
|
func extractEXIFWithDatetime(data []byte) (json.RawMessage, *time.Time) {
|
||||||
|
x, err := exif.Decode(bytes.NewReader(data))
|
||||||
|
if err != nil {
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
b, err := x.MarshalJSON()
|
||||||
|
if err != nil {
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
var dt *time.Time
|
||||||
|
if t, err := x.DateTime(); err == nil {
|
||||||
|
dt = &t
|
||||||
|
}
|
||||||
|
return json.RawMessage(b), dt
|
||||||
|
}
|
||||||
@@ -0,0 +1,110 @@
|
|||||||
|
package service
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"encoding/json"
|
||||||
|
"image"
|
||||||
|
"image/color"
|
||||||
|
"image/png"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestParseExifDate(t *testing.T) {
|
||||||
|
cases := []struct {
|
||||||
|
in string
|
||||||
|
ok bool
|
||||||
|
want time.Time
|
||||||
|
}{
|
||||||
|
{"2026:03:24 16:57:58", true, time.Date(2026, 3, 24, 16, 57, 58, 0, time.UTC)},
|
||||||
|
{"2026:05:08 23:07:55+03:00", true, time.Date(2026, 5, 8, 23, 7, 55, 0, time.FixedZone("", 3*3600))},
|
||||||
|
{" 2026:01:02 03:04:05 ", true, time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC)},
|
||||||
|
{"0000:00:00 00:00:00", false, time.Time{}},
|
||||||
|
{"not a date", false, time.Time{}},
|
||||||
|
{"", false, time.Time{}},
|
||||||
|
}
|
||||||
|
for _, c := range cases {
|
||||||
|
got, ok := parseExifDate(c.in)
|
||||||
|
if ok != c.ok {
|
||||||
|
t.Errorf("parseExifDate(%q) ok=%v, want %v", c.in, ok, c.ok)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if ok && !got.Equal(c.want) {
|
||||||
|
t.Errorf("parseExifDate(%q) = %v, want %v", c.in, got, c.want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// tinyPNG returns a valid 2x3 PNG with no embedded EXIF/date.
|
||||||
|
func tinyPNG(t *testing.T) []byte {
|
||||||
|
t.Helper()
|
||||||
|
img := image.NewRGBA(image.Rect(0, 0, 2, 3))
|
||||||
|
img.Set(0, 0, color.RGBA{R: 10, G: 20, B: 30, A: 255})
|
||||||
|
var buf bytes.Buffer
|
||||||
|
if err := png.Encode(&buf, img); err != nil {
|
||||||
|
t.Fatalf("encode png: %v", err)
|
||||||
|
}
|
||||||
|
return buf.Bytes()
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExtractMetadataExiftool(t *testing.T) {
|
||||||
|
if exiftoolPath == "" {
|
||||||
|
t.Skip("exiftool not installed; metadata extraction falls back to goexif")
|
||||||
|
}
|
||||||
|
|
||||||
|
raw, dt := extractMetadata(tinyPNG(t), "snapshot.png", nil)
|
||||||
|
if raw == nil {
|
||||||
|
t.Fatal("expected non-nil metadata JSON")
|
||||||
|
}
|
||||||
|
if dt != nil {
|
||||||
|
t.Errorf("a PNG without a capture date should yield no content datetime, got %v", dt)
|
||||||
|
}
|
||||||
|
|
||||||
|
var m map[string]json.RawMessage
|
||||||
|
if err := json.Unmarshal(raw, &m); err != nil {
|
||||||
|
t.Fatalf("metadata is not valid JSON: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// exiftool understood the format (goexif never would for PNG).
|
||||||
|
if v := jsonString(t, m, "FileType"); v != "PNG" {
|
||||||
|
t.Errorf("FileType = %q, want PNG", v)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Dimensions are numeric, not human-readable strings.
|
||||||
|
for _, key := range []string{"ImageWidth", "ImageHeight"} {
|
||||||
|
raw, ok := m[key]
|
||||||
|
if !ok {
|
||||||
|
t.Errorf("missing %s", key)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
var n float64
|
||||||
|
if err := json.Unmarshal(raw, &n); err != nil {
|
||||||
|
t.Errorf("%s is not numeric: %s", key, raw)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// FileName is the original, not the temp file; temp-file artifacts are gone.
|
||||||
|
if v := jsonString(t, m, "FileName"); v != "snapshot.png" {
|
||||||
|
t.Errorf("FileName = %q, want snapshot.png", v)
|
||||||
|
}
|
||||||
|
for _, leaked := range []string{"SourceFile", "Directory", "FilePermissions", "FileModifyDate"} {
|
||||||
|
if _, ok := m[leaked]; ok {
|
||||||
|
t.Errorf("temp-file field %q should have been stripped", leaked)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func jsonString(t *testing.T, m map[string]json.RawMessage, key string) string {
|
||||||
|
t.Helper()
|
||||||
|
raw, ok := m[key]
|
||||||
|
if !ok {
|
||||||
|
t.Errorf("missing key %q", key)
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
var s string
|
||||||
|
if err := json.Unmarshal(raw, &s); err != nil {
|
||||||
|
t.Errorf("key %q is not a string: %s", key, raw)
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
return s
|
||||||
|
}
|
||||||
@@ -10,15 +10,18 @@ import (
|
|||||||
"tanabata/backend/internal/port"
|
"tanabata/backend/internal/port"
|
||||||
)
|
)
|
||||||
|
|
||||||
const poolObjectType = "pool"
|
const poolObjectType = "pool"
|
||||||
const poolObjectTypeID int16 = 4 // fourth row in 007_seed_data.sql object_types
|
const poolObjectTypeID int16 = 4 // fourth row in 007_seed_data.sql object_types
|
||||||
|
|
||||||
// PoolParams holds the fields for creating or patching a pool.
|
// PoolParams holds the fields for creating or patching a pool. SortKey and
|
||||||
|
// SortOrder are pointers so a patch can leave them unchanged (nil) vs set them.
|
||||||
type PoolParams struct {
|
type PoolParams struct {
|
||||||
Name string
|
Name string
|
||||||
Notes *string
|
Notes *string
|
||||||
Metadata json.RawMessage
|
Metadata json.RawMessage
|
||||||
IsPublic *bool
|
IsPublic *bool
|
||||||
|
SortKey *string
|
||||||
|
SortOrder *string
|
||||||
}
|
}
|
||||||
|
|
||||||
// PoolService handles pool CRUD and pool–file management with ACL + audit.
|
// PoolService handles pool CRUD and pool–file management with ACL + audit.
|
||||||
@@ -41,14 +44,73 @@ func NewPoolService(
|
|||||||
// CRUD
|
// CRUD
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
// List returns a paginated list of pools.
|
// List returns a paginated list of pools the caller may see.
|
||||||
func (s *PoolService) List(ctx context.Context, params port.OffsetParams) (*domain.PoolOffsetPage, error) {
|
func (s *PoolService) List(ctx context.Context, params port.OffsetParams) (*domain.PoolOffsetPage, error) {
|
||||||
|
params.ViewerID, params.ViewerIsAdmin, _ = domain.UserFromContext(ctx)
|
||||||
return s.pools.List(ctx, params)
|
return s.pools.List(ctx, params)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Get returns a pool by ID.
|
// Get returns a pool by ID, enforcing view ACL.
|
||||||
func (s *PoolService) Get(ctx context.Context, id uuid.UUID) (*domain.Pool, error) {
|
func (s *PoolService) Get(ctx context.Context, id uuid.UUID) (*domain.Pool, error) {
|
||||||
return s.pools.GetByID(ctx, id)
|
userID, isAdmin, _ := domain.UserFromContext(ctx)
|
||||||
|
p, err := s.pools.GetByID(ctx, id)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
ok, err := s.acl.CanView(ctx, userID, isAdmin, p.CreatorID, p.IsPublic, poolObjectTypeID, id)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if !ok {
|
||||||
|
return nil, domain.ErrForbidden
|
||||||
|
}
|
||||||
|
return p, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// authorizeView returns nil if the caller may view the pool, else ErrForbidden
|
||||||
|
// (or ErrNotFound if the pool does not exist).
|
||||||
|
func (s *PoolService) authorizeView(ctx context.Context, poolID uuid.UUID) error {
|
||||||
|
userID, isAdmin, _ := domain.UserFromContext(ctx)
|
||||||
|
p, err := s.pools.GetByID(ctx, poolID)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
ok, err := s.acl.CanView(ctx, userID, isAdmin, p.CreatorID, p.IsPublic, poolObjectTypeID, poolID)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if !ok {
|
||||||
|
return domain.ErrForbidden
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// RecordView appends a view-history entry for the current user, enforcing view
|
||||||
|
// ACL (you can only record a view of a pool you may see).
|
||||||
|
func (s *PoolService) RecordView(ctx context.Context, id uuid.UUID) error {
|
||||||
|
userID, _, _ := domain.UserFromContext(ctx)
|
||||||
|
if err := s.authorizeView(ctx, id); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return s.pools.RecordView(ctx, id, userID)
|
||||||
|
}
|
||||||
|
|
||||||
|
// authorizeEdit returns nil if the caller may edit the pool, else ErrForbidden
|
||||||
|
// (or ErrNotFound if the pool does not exist).
|
||||||
|
func (s *PoolService) authorizeEdit(ctx context.Context, poolID uuid.UUID) error {
|
||||||
|
userID, isAdmin, _ := domain.UserFromContext(ctx)
|
||||||
|
p, err := s.pools.GetByID(ctx, poolID)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
ok, err := s.acl.CanEdit(ctx, userID, isAdmin, p.CreatorID, poolObjectTypeID, poolID)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if !ok {
|
||||||
|
return domain.ErrForbidden
|
||||||
|
}
|
||||||
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// Create inserts a new pool.
|
// Create inserts a new pool.
|
||||||
@@ -105,6 +167,18 @@ func (s *PoolService) Update(ctx context.Context, id uuid.UUID, p PoolParams) (*
|
|||||||
if p.IsPublic != nil {
|
if p.IsPublic != nil {
|
||||||
patch.IsPublic = *p.IsPublic
|
patch.IsPublic = *p.IsPublic
|
||||||
}
|
}
|
||||||
|
if p.SortKey != nil {
|
||||||
|
if !domain.ValidPoolSortKey(*p.SortKey) {
|
||||||
|
return nil, domain.ErrValidation
|
||||||
|
}
|
||||||
|
patch.SortKey = *p.SortKey
|
||||||
|
}
|
||||||
|
if p.SortOrder != nil {
|
||||||
|
if !domain.ValidSortOrder(*p.SortOrder) {
|
||||||
|
return nil, domain.ErrValidation
|
||||||
|
}
|
||||||
|
patch.SortOrder = *p.SortOrder
|
||||||
|
}
|
||||||
|
|
||||||
updated, err := s.pools.Update(ctx, id, &patch)
|
updated, err := s.pools.Update(ctx, id, &patch)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -146,13 +220,33 @@ func (s *PoolService) Delete(ctx context.Context, id uuid.UUID) error {
|
|||||||
// Pool–file operations
|
// Pool–file operations
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
// ListFiles returns cursor-paginated files within a pool ordered by position.
|
// ListFiles returns cursor-paginated files within a pool, enforcing view ACL.
|
||||||
|
// The ordering is the pool's own stored sort setting (manual position order or
|
||||||
|
// an automatic file-field sort), not a request parameter.
|
||||||
func (s *PoolService) ListFiles(ctx context.Context, poolID uuid.UUID, params port.PoolFileListParams) (*domain.PoolFilePage, error) {
|
func (s *PoolService) ListFiles(ctx context.Context, poolID uuid.UUID, params port.PoolFileListParams) (*domain.PoolFilePage, error) {
|
||||||
|
userID, isAdmin, _ := domain.UserFromContext(ctx)
|
||||||
|
pool, err := s.pools.GetByID(ctx, poolID)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
ok, err := s.acl.CanView(ctx, userID, isAdmin, pool.CreatorID, pool.IsPublic, poolObjectTypeID, poolID)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if !ok {
|
||||||
|
return nil, domain.ErrForbidden
|
||||||
|
}
|
||||||
|
params.SortKey = pool.SortKey
|
||||||
|
params.SortOrder = pool.SortOrder
|
||||||
return s.pools.ListFiles(ctx, poolID, params)
|
return s.pools.ListFiles(ctx, poolID, params)
|
||||||
}
|
}
|
||||||
|
|
||||||
// AddFiles adds files to a pool at the given position (nil = append).
|
// AddFiles adds files to a pool at the given position (nil = append), enforcing
|
||||||
|
// edit ACL on the pool.
|
||||||
func (s *PoolService) AddFiles(ctx context.Context, poolID uuid.UUID, fileIDs []uuid.UUID, position *int) error {
|
func (s *PoolService) AddFiles(ctx context.Context, poolID uuid.UUID, fileIDs []uuid.UUID, position *int) error {
|
||||||
|
if err := s.authorizeEdit(ctx, poolID); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
if err := s.pools.AddFiles(ctx, poolID, fileIDs, position); err != nil {
|
if err := s.pools.AddFiles(ctx, poolID, fileIDs, position); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
@@ -161,8 +255,11 @@ func (s *PoolService) AddFiles(ctx context.Context, poolID uuid.UUID, fileIDs []
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// RemoveFiles removes files from a pool.
|
// RemoveFiles removes files from a pool, enforcing edit ACL on the pool.
|
||||||
func (s *PoolService) RemoveFiles(ctx context.Context, poolID uuid.UUID, fileIDs []uuid.UUID) error {
|
func (s *PoolService) RemoveFiles(ctx context.Context, poolID uuid.UUID, fileIDs []uuid.UUID) error {
|
||||||
|
if err := s.authorizeEdit(ctx, poolID); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
if err := s.pools.RemoveFiles(ctx, poolID, fileIDs); err != nil {
|
if err := s.pools.RemoveFiles(ctx, poolID, fileIDs); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
@@ -171,7 +268,29 @@ func (s *PoolService) RemoveFiles(ctx context.Context, poolID uuid.UUID, fileIDs
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// Reorder sets the full ordered sequence of file IDs within a pool.
|
// Reorder sets the ordered sequence of file IDs within a pool, enforcing edit
|
||||||
|
// ACL on the pool. Manual ordering only applies when the pool's sort key is
|
||||||
|
// "manual"; reordering an auto-sorted pool is rejected as a validation error.
|
||||||
func (s *PoolService) Reorder(ctx context.Context, poolID uuid.UUID, fileIDs []uuid.UUID) error {
|
func (s *PoolService) Reorder(ctx context.Context, poolID uuid.UUID, fileIDs []uuid.UUID) error {
|
||||||
return s.pools.Reorder(ctx, poolID, fileIDs)
|
userID, isAdmin, _ := domain.UserFromContext(ctx)
|
||||||
}
|
pool, err := s.pools.GetByID(ctx, poolID)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
ok, err := s.acl.CanEdit(ctx, userID, isAdmin, pool.CreatorID, poolObjectTypeID, poolID)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if !ok {
|
||||||
|
return domain.ErrForbidden
|
||||||
|
}
|
||||||
|
if pool.SortKey != domain.PoolSortManual {
|
||||||
|
return domain.ErrValidation
|
||||||
|
}
|
||||||
|
if err := s.pools.Reorder(ctx, poolID, fileIDs); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
objType := poolObjectType
|
||||||
|
_ = s.audit.Log(ctx, "file_pool_reorder", &objType, &poolID, map[string]any{"count": len(fileIDs)})
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|||||||
@@ -10,15 +10,15 @@ import (
|
|||||||
"tanabata/backend/internal/port"
|
"tanabata/backend/internal/port"
|
||||||
)
|
)
|
||||||
|
|
||||||
const tagObjectType = "tag"
|
const tagObjectType = "tag"
|
||||||
const tagObjectTypeID int16 = 2 // second row in 007_seed_data.sql object_types
|
const tagObjectTypeID int16 = 2 // second row in 007_seed_data.sql object_types
|
||||||
|
|
||||||
// TagParams holds the fields for creating or patching a tag.
|
// TagParams holds the fields for creating or patching a tag.
|
||||||
type TagParams struct {
|
type TagParams struct {
|
||||||
Name string
|
Name string
|
||||||
Notes *string
|
Notes *string
|
||||||
Color *string // nil = no change; pointer to empty string = clear
|
Color *string // nil = no change; pointer to empty string = clear
|
||||||
CategoryID *uuid.UUID // nil = no change; Nil UUID = unassign
|
CategoryID *uuid.UUID // nil = no change; Nil UUID = unassign
|
||||||
Metadata json.RawMessage
|
Metadata json.RawMessage
|
||||||
IsPublic *bool
|
IsPublic *bool
|
||||||
}
|
}
|
||||||
@@ -54,14 +54,27 @@ func NewTagService(
|
|||||||
// Tag CRUD
|
// Tag CRUD
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
// List returns a paginated, optionally filtered list of tags.
|
// List returns a paginated, optionally filtered list of tags the caller may see.
|
||||||
func (s *TagService) List(ctx context.Context, params port.OffsetParams) (*domain.TagOffsetPage, error) {
|
func (s *TagService) List(ctx context.Context, params port.OffsetParams) (*domain.TagOffsetPage, error) {
|
||||||
|
params.ViewerID, params.ViewerIsAdmin, _ = domain.UserFromContext(ctx)
|
||||||
return s.tags.List(ctx, params)
|
return s.tags.List(ctx, params)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Get returns a tag by ID.
|
// Get returns a tag by ID, enforcing view ACL.
|
||||||
func (s *TagService) Get(ctx context.Context, id uuid.UUID) (*domain.Tag, error) {
|
func (s *TagService) Get(ctx context.Context, id uuid.UUID) (*domain.Tag, error) {
|
||||||
return s.tags.GetByID(ctx, id)
|
userID, isAdmin, _ := domain.UserFromContext(ctx)
|
||||||
|
t, err := s.tags.GetByID(ctx, id)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
ok, err := s.acl.CanView(ctx, userID, isAdmin, t.CreatorID, t.IsPublic, tagObjectTypeID, id)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if !ok {
|
||||||
|
return nil, domain.ErrForbidden
|
||||||
|
}
|
||||||
|
return t, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// Create inserts a new tag record.
|
// Create inserts a new tag record.
|
||||||
@@ -179,16 +192,32 @@ func (s *TagService) ListRules(ctx context.Context, tagID uuid.UUID) ([]domain.T
|
|||||||
return s.rules.ListByTag(ctx, tagID)
|
return s.rules.ListByTag(ctx, tagID)
|
||||||
}
|
}
|
||||||
|
|
||||||
// CreateRule adds a tag rule. If applyToExisting is true, the then_tag is
|
// CreateRule adds a tag rule. When the rule is active and applyToExisting is
|
||||||
// retroactively applied to all files that already carry the when_tag.
|
// true, the full transitive expansion of thenTagID is retroactively applied to
|
||||||
// Retroactive application requires a FileRepo; it is deferred until wired
|
// every file already carrying whenTagID — same semantics as activating an
|
||||||
// in a future iteration (see port.FileRepo.ListByTag).
|
// existing rule via SetRuleActive. The insert and retroactive apply run in one
|
||||||
func (s *TagService) CreateRule(ctx context.Context, whenTagID, thenTagID uuid.UUID, isActive, _ bool) (*domain.TagRule, error) {
|
// transaction so a file is never left half-tagged.
|
||||||
return s.rules.Create(ctx, domain.TagRule{
|
func (s *TagService) CreateRule(ctx context.Context, whenTagID, thenTagID uuid.UUID, isActive, applyToExisting bool) (*domain.TagRule, error) {
|
||||||
WhenTagID: whenTagID,
|
var created *domain.TagRule
|
||||||
ThenTagID: thenTagID,
|
err := s.tx.WithTx(ctx, func(ctx context.Context) error {
|
||||||
IsActive: isActive,
|
rule, err := s.rules.Create(ctx, domain.TagRule{
|
||||||
|
WhenTagID: whenTagID,
|
||||||
|
ThenTagID: thenTagID,
|
||||||
|
IsActive: isActive,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
created = rule
|
||||||
|
if isActive && applyToExisting {
|
||||||
|
return s.rules.ApplyToExisting(ctx, whenTagID, thenTagID)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
})
|
})
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return created, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// SetRuleActive toggles a rule's is_active flag and returns the updated rule.
|
// SetRuleActive toggles a rule's is_active flag and returns the updated rule.
|
||||||
@@ -409,4 +438,4 @@ func (s *TagService) expandTagSet(ctx context.Context, seeds []uuid.UUID) ([]uui
|
|||||||
}
|
}
|
||||||
|
|
||||||
return queue, nil
|
return queue, nil
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -189,4 +189,4 @@ func (s *UserService) Delete(ctx context.Context, id int16) error {
|
|||||||
}
|
}
|
||||||
_ = s.audit.Log(ctx, "user_delete", nil, nil, map[string]any{"target_user_id": id})
|
_ = s.audit.Log(ctx, "user_delete", nil, nil, map[string]any{"target_user_id": id})
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -5,16 +5,19 @@ import (
|
|||||||
"bytes"
|
"bytes"
|
||||||
"context"
|
"context"
|
||||||
"fmt"
|
"fmt"
|
||||||
|
_ "golang.org/x/image/webp" // register WebP decoder
|
||||||
"image"
|
"image"
|
||||||
"image/color"
|
"image/color"
|
||||||
|
_ "image/gif" // register GIF decoder
|
||||||
"image/jpeg"
|
"image/jpeg"
|
||||||
_ "image/gif" // register GIF decoder
|
_ "image/png" // register PNG decoder
|
||||||
_ "image/png" // register PNG decoder
|
|
||||||
_ "golang.org/x/image/webp" // register WebP decoder
|
|
||||||
"io"
|
"io"
|
||||||
"os"
|
"os"
|
||||||
"os/exec"
|
"os/exec"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
|
"runtime"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/disintegration/imaging"
|
"github.com/disintegration/imaging"
|
||||||
@@ -38,20 +41,36 @@ type DiskStorage struct {
|
|||||||
thumbHeight int
|
thumbHeight int
|
||||||
previewWidth int
|
previewWidth int
|
||||||
previewHeight int
|
previewHeight int
|
||||||
|
maxPixels int
|
||||||
|
// genSem bounds concurrent thumbnail/preview generation. Each resize already
|
||||||
|
// fans out across every core (imaging uses GOMAXPROCS), and large sources cost
|
||||||
|
// hundreds of MB to decode, so unbounded parallelism on a burst of big images
|
||||||
|
// pegs the CPU and can exhaust RAM. A buffered channel caps how many run at once.
|
||||||
|
genSem chan struct{}
|
||||||
}
|
}
|
||||||
|
|
||||||
var _ port.FileStorage = (*DiskStorage)(nil)
|
var _ port.FileStorage = (*DiskStorage)(nil)
|
||||||
|
|
||||||
// NewDiskStorage creates a DiskStorage and ensures both directories exist.
|
// NewDiskStorage creates a DiskStorage and ensures both directories exist.
|
||||||
|
//
|
||||||
|
// maxPixels caps the source pixel count we will decode in-process (0 → a sane
|
||||||
|
// default). concurrency bounds simultaneous generation (≤0 → half the CPUs).
|
||||||
func NewDiskStorage(
|
func NewDiskStorage(
|
||||||
filesPath, thumbsPath string,
|
filesPath, thumbsPath string,
|
||||||
thumbW, thumbH, prevW, prevH int,
|
thumbW, thumbH, prevW, prevH int,
|
||||||
|
maxPixels, concurrency int,
|
||||||
) (*DiskStorage, error) {
|
) (*DiskStorage, error) {
|
||||||
for _, p := range []string{filesPath, thumbsPath} {
|
for _, p := range []string{filesPath, thumbsPath} {
|
||||||
if err := os.MkdirAll(p, 0o755); err != nil {
|
if err := os.MkdirAll(p, 0o755); err != nil {
|
||||||
return nil, fmt.Errorf("storage: create directory %q: %w", p, err)
|
return nil, fmt.Errorf("storage: create directory %q: %w", p, err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
if maxPixels <= 0 {
|
||||||
|
maxPixels = defaultMaxDecodePixels
|
||||||
|
}
|
||||||
|
if concurrency <= 0 {
|
||||||
|
concurrency = max(1, runtime.GOMAXPROCS(0)/2)
|
||||||
|
}
|
||||||
return &DiskStorage{
|
return &DiskStorage{
|
||||||
filesPath: filesPath,
|
filesPath: filesPath,
|
||||||
thumbsPath: thumbsPath,
|
thumbsPath: thumbsPath,
|
||||||
@@ -59,6 +78,8 @@ func NewDiskStorage(
|
|||||||
thumbHeight: thumbH,
|
thumbHeight: thumbH,
|
||||||
previewWidth: prevW,
|
previewWidth: prevW,
|
||||||
previewHeight: prevH,
|
previewHeight: prevH,
|
||||||
|
maxPixels: maxPixels,
|
||||||
|
genSem: make(chan struct{}, concurrency),
|
||||||
}, nil
|
}, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -120,32 +141,55 @@ func (s *DiskStorage) InvalidateCache(_ context.Context, id uuid.UUID) error {
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// Thumbnail returns a JPEG that fits within the configured max width×height
|
// Thumbnail returns a JPEG scaled to fit within the configured max width×height,
|
||||||
// (never upscaled, never cropped). Generated on first call and cached.
|
// preserving the original aspect ratio (never upscaled, never cropped); the grid
|
||||||
// Video files are thumbnailed via ffmpeg; other non-image files get a placeholder.
|
// cell letterboxes it as needed. Generated on first call and cached. Video files
|
||||||
|
// are thumbnailed via ffmpeg; other non-image files get a placeholder.
|
||||||
func (s *DiskStorage) Thumbnail(ctx context.Context, id uuid.UUID) (io.ReadCloser, error) {
|
func (s *DiskStorage) Thumbnail(ctx context.Context, id uuid.UUID) (io.ReadCloser, error) {
|
||||||
return s.serveGenerated(ctx, id, s.thumbCachePath(id), s.thumbWidth, s.thumbHeight)
|
return s.serveGenerated(ctx, id, s.thumbCachePath(id), s.thumbWidth, s.thumbHeight)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Preview returns a JPEG that fits within the configured max width×height
|
// Preview returns a JPEG scaled to fit within the configured max width×height,
|
||||||
// (never upscaled, never cropped). Generated on first call and cached.
|
// preserving the original aspect ratio (never upscaled, never cropped) so the
|
||||||
// Video files are thumbnailed via ffmpeg; other non-image files get a placeholder.
|
// viewer shows the whole image. Generated on first call and cached. Video files
|
||||||
|
// are thumbnailed via ffmpeg; other non-image files get a placeholder.
|
||||||
func (s *DiskStorage) Preview(ctx context.Context, id uuid.UUID) (io.ReadCloser, error) {
|
func (s *DiskStorage) Preview(ctx context.Context, id uuid.UUID) (io.ReadCloser, error) {
|
||||||
return s.serveGenerated(ctx, id, s.previewCachePath(id), s.previewWidth, s.previewHeight)
|
return s.serveGenerated(ctx, id, s.previewCachePath(id), s.previewWidth, s.previewHeight)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// VideoFrameMiddle decodes a representative frame from the middle of a video
|
||||||
|
// (duration/2). The midpoint avoids the shared intros, title cards and black
|
||||||
|
// lead-in frames that make a fixed early offset collide across unrelated clips,
|
||||||
|
// so it is the right source for the video's perceptual (duplicate-detection)
|
||||||
|
// hash. The file must already exist in storage; ffmpeg/ffprobe must be installed.
|
||||||
|
// This is not part of port.FileStorage — only the dedup CLI needs it, with a
|
||||||
|
// concrete *DiskStorage — so the interface stays lean and ffmpeg stays out of the
|
||||||
|
// upload path.
|
||||||
|
func (s *DiskStorage) VideoFrameMiddle(ctx context.Context, id uuid.UUID) (image.Image, error) {
|
||||||
|
srcPath := s.originalPath(id)
|
||||||
|
if _, err := os.Stat(srcPath); err != nil {
|
||||||
|
if os.IsNotExist(err) {
|
||||||
|
return nil, domain.ErrNotFound
|
||||||
|
}
|
||||||
|
return nil, fmt.Errorf("storage: stat %q: %w", srcPath, err)
|
||||||
|
}
|
||||||
|
return extractVideoFrameMiddle(ctx, srcPath)
|
||||||
|
}
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// Internal helpers
|
// Internal helpers
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
// serveGenerated is the shared implementation for Thumbnail and Preview.
|
// serveGenerated is the shared implementation for Thumbnail and Preview. Both
|
||||||
// imaging.Thumbnail fits the source within maxW×maxH without upscaling or cropping.
|
// fit the source within maxW×maxH preserving the aspect ratio (no crop, no
|
||||||
|
// upscale); they differ only in the configured dimensions.
|
||||||
//
|
//
|
||||||
// Resolution order:
|
// Resolution order:
|
||||||
// 1. Return cached JPEG if present.
|
// 1. Return cached JPEG if present.
|
||||||
// 2. Decode as still image (JPEG/PNG/GIF via imaging).
|
// 2. vipsthumbnail (shrink-on-load; the primary still-image path).
|
||||||
// 3. Extract a frame with ffmpeg (video files).
|
// 3. Pure-Go decode + imaging.Fit (fallback when vips is absent).
|
||||||
// 4. Solid-colour placeholder (archives, unrecognised formats, etc.).
|
// 4. Extract a frame with ffmpeg (video files).
|
||||||
|
// 5. Solid-colour placeholder (archives, unrecognised formats, etc.).
|
||||||
func (s *DiskStorage) serveGenerated(ctx context.Context, id uuid.UUID, cachePath string, maxW, maxH int) (io.ReadCloser, error) {
|
func (s *DiskStorage) serveGenerated(ctx context.Context, id uuid.UUID, cachePath string, maxW, maxH int) (io.ReadCloser, error) {
|
||||||
// Fast path: cache hit.
|
// Fast path: cache hit.
|
||||||
if f, err := os.Open(cachePath); err == nil {
|
if f, err := os.Open(cachePath); err == nil {
|
||||||
@@ -161,14 +205,40 @@ func (s *DiskStorage) serveGenerated(ctx context.Context, id uuid.UUID, cachePat
|
|||||||
return nil, fmt.Errorf("storage: stat %q: %w", srcPath, err)
|
return nil, fmt.Errorf("storage: stat %q: %w", srcPath, err)
|
||||||
}
|
}
|
||||||
|
|
||||||
// 1. Try still-image decode (JPEG/PNG/GIF), rejecting decompression bombs.
|
// Bound concurrent generation so a burst of large images can't peg every core
|
||||||
// 2. Try video frame extraction via ffmpeg.
|
// or exhaust RAM. Queue here (respecting cancellation) rather than starting
|
||||||
// 3. Fall back to placeholder.
|
// the heavy decode immediately.
|
||||||
|
select {
|
||||||
|
case s.genSem <- struct{}{}:
|
||||||
|
defer func() { <-s.genSem }()
|
||||||
|
case <-ctx.Done():
|
||||||
|
return nil, ctx.Err()
|
||||||
|
}
|
||||||
|
|
||||||
|
// Another request may have generated this while we waited on the semaphore.
|
||||||
|
if f, err := os.Open(cachePath); err == nil {
|
||||||
|
return f, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Primary path: vipsthumbnail. It shrinks on load (e.g. JPEG DCT scaling), so
|
||||||
|
// even a 200+ Mpx photo is thumbnailed in a fraction of the memory and CPU of a
|
||||||
|
// full in-process decode, writing the final JPEG straight to the cache. Falls
|
||||||
|
// through when vips is absent or can't read the source (e.g. a video).
|
||||||
|
if vipsThumbnailPath != "" {
|
||||||
|
if rc, err := s.vipsThumbnail(ctx, srcPath, cachePath, maxW, maxH); err == nil {
|
||||||
|
return rc, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Fallback pipeline (pure Go):
|
||||||
|
// 1. Still-image decode (JPEG/PNG/GIF), rejecting oversized rasters.
|
||||||
|
// 2. Video frame extraction via ffmpeg.
|
||||||
|
// 3. Solid-colour placeholder.
|
||||||
var img image.Image
|
var img image.Image
|
||||||
if decoded, err := decodeImageLimited(srcPath); err == nil {
|
if decoded, err := decodeImageLimited(srcPath, s.maxPixels); err == nil {
|
||||||
img = imaging.Thumbnail(decoded, maxW, maxH, imaging.Lanczos)
|
img = imaging.Fit(decoded, maxW, maxH, imaging.Lanczos)
|
||||||
} else if frame, err := extractVideoFrame(ctx, srcPath); err == nil {
|
} else if frame, err := extractVideoFrameMiddle(ctx, srcPath); err == nil {
|
||||||
img = imaging.Thumbnail(frame, maxW, maxH, imaging.Lanczos)
|
img = imaging.Fit(frame, maxW, maxH, imaging.Lanczos)
|
||||||
} else {
|
} else {
|
||||||
img = placeholder(maxW, maxH)
|
img = placeholder(maxW, maxH)
|
||||||
}
|
}
|
||||||
@@ -218,15 +288,15 @@ func writeCache(cachePath string, img image.Image) (io.ReadCloser, error) {
|
|||||||
return f, nil
|
return f, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// maxDecodePixels caps the pixel count of an image we are willing to decode
|
// defaultMaxDecodePixels is the fallback cap when none is configured. It bounds
|
||||||
// into memory, bounding the cost of a decompression bomb (a tiny file that
|
// the cost of a decompression bomb (a tiny file that expands to an enormous
|
||||||
// expands to an enormous raster). 64 Mpx is ~ an 8192×8192 image.
|
// raster) and the per-image memory; ~300 Mpx covers e.g. a 13000×17000 photo.
|
||||||
const maxDecodePixels = 64 << 20
|
const defaultMaxDecodePixels = 300_000_000
|
||||||
|
|
||||||
// decodeImageLimited decodes the image at path after first inspecting its header
|
// decodeImageLimited decodes the image at path after first inspecting its header
|
||||||
// dimensions via image.DecodeConfig (which does not allocate the raster), and
|
// dimensions via image.DecodeConfig (which does not allocate the raster), and
|
||||||
// refuses images whose pixel count exceeds maxDecodePixels.
|
// refuses images whose pixel count exceeds maxPixels.
|
||||||
func decodeImageLimited(path string) (image.Image, error) {
|
func decodeImageLimited(path string, maxPixels int) (image.Image, error) {
|
||||||
f, err := os.Open(path)
|
f, err := os.Open(path)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
@@ -237,7 +307,7 @@ func decodeImageLimited(path string) (image.Image, error) {
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
if int64(cfg.Width)*int64(cfg.Height) > maxDecodePixels {
|
if int64(cfg.Width)*int64(cfg.Height) > int64(maxPixels) {
|
||||||
return nil, fmt.Errorf("image too large to decode: %dx%d", cfg.Width, cfg.Height)
|
return nil, fmt.Errorf("image too large to decode: %dx%d", cfg.Width, cfg.Height)
|
||||||
}
|
}
|
||||||
if _, err := f.Seek(0, io.SeekStart); err != nil {
|
if _, err := f.Seek(0, io.SeekStart); err != nil {
|
||||||
@@ -246,19 +316,80 @@ func decodeImageLimited(path string) (image.Image, error) {
|
|||||||
return imaging.Decode(f, imaging.AutoOrientation(true))
|
return imaging.Decode(f, imaging.AutoOrientation(true))
|
||||||
}
|
}
|
||||||
|
|
||||||
// extractVideoFrame uses ffmpeg to extract a single frame from a video file.
|
// vipsThumbnailPath is the resolved path to the vipsthumbnail CLI, or "" when it
|
||||||
// It seeks 1 second in (keyframe-accurate fast seek) and pipes the frame out
|
// isn't installed — in which case generation falls back to the pure-Go pipeline.
|
||||||
// as PNG. If the video is shorter than 1 s the seek is silently ignored by
|
var vipsThumbnailPath, _ = exec.LookPath("vipsthumbnail")
|
||||||
// ffmpeg and the first available frame is returned instead.
|
|
||||||
// Returns an error if ffmpeg is not installed or produces no output. The run is
|
// vipsThumbnail generates a JPEG thumbnail with the vipsthumbnail CLI, writing it
|
||||||
// bounded by a timeout so a malformed file cannot hang the request indefinitely.
|
// straight to cachePath via an atomic temp→rename. vips decodes large images at a
|
||||||
func extractVideoFrame(ctx context.Context, srcPath string) (image.Image, error) {
|
// reduced scale (shrink-on-load), so this costs a fraction of the memory and CPU
|
||||||
|
// of a full in-process decode. The result is fit within maxW×maxH and never
|
||||||
|
// upscaled (the ">" size modifier). Returns an error for inputs vips can't read
|
||||||
|
// (e.g. videos) so the caller can fall back.
|
||||||
|
func (s *DiskStorage) vipsThumbnail(ctx context.Context, srcPath, cachePath string, maxW, maxH int) (io.ReadCloser, error) {
|
||||||
|
ctx, cancel := context.WithTimeout(ctx, 30*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
|
||||||
|
tmp, err := os.CreateTemp(filepath.Dir(cachePath), ".vips-*.jpg")
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("storage: create temp file: %w", err)
|
||||||
|
}
|
||||||
|
tmpName := tmp.Name()
|
||||||
|
_ = tmp.Close()
|
||||||
|
|
||||||
|
cmd := exec.CommandContext(ctx, vipsThumbnailPath,
|
||||||
|
srcPath,
|
||||||
|
"--size", fmt.Sprintf("%dx%d>", maxW, maxH),
|
||||||
|
"--output", tmpName+"[Q=85]",
|
||||||
|
)
|
||||||
|
cmd.Stderr = io.Discard
|
||||||
|
|
||||||
|
if err := cmd.Run(); err != nil {
|
||||||
|
os.Remove(tmpName)
|
||||||
|
return nil, fmt.Errorf("vipsthumbnail: %w", err)
|
||||||
|
}
|
||||||
|
if fi, err := os.Stat(tmpName); err != nil || fi.Size() == 0 {
|
||||||
|
os.Remove(tmpName)
|
||||||
|
return nil, fmt.Errorf("vipsthumbnail: no output produced")
|
||||||
|
}
|
||||||
|
if err := os.Rename(tmpName, cachePath); err != nil {
|
||||||
|
os.Remove(tmpName)
|
||||||
|
return nil, fmt.Errorf("storage: rename cache file: %w", err)
|
||||||
|
}
|
||||||
|
f, err := os.Open(cachePath)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("storage: open cache file: %w", err)
|
||||||
|
}
|
||||||
|
return f, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// extractVideoFrameMiddle extracts a single frame from the middle of the video
|
||||||
|
// (duration/2), falling back to a 1s offset when the duration can't be probed.
|
||||||
|
// The midpoint dodges shared intros, title cards and black lead-in frames, and
|
||||||
|
// matches the frame used for the perceptual hash so a video's thumbnail/preview
|
||||||
|
// shows the same representative frame dedup compared. See extractVideoFrameAt for
|
||||||
|
// the mechanics.
|
||||||
|
func extractVideoFrameMiddle(ctx context.Context, srcPath string) (image.Image, error) {
|
||||||
|
at := 1.0
|
||||||
|
if d, err := videoDurationSeconds(ctx, srcPath); err == nil && d > 0 {
|
||||||
|
at = d / 2
|
||||||
|
}
|
||||||
|
return extractVideoFrameAt(ctx, srcPath, at)
|
||||||
|
}
|
||||||
|
|
||||||
|
// extractVideoFrameAt uses ffmpeg to extract a single frame at atSec seconds into
|
||||||
|
// the video, piped out as PNG. The fast input seek (-ss before -i) is keyframe-
|
||||||
|
// accurate and cheap; if atSec is past the end the seek is silently ignored and
|
||||||
|
// the first available frame is returned instead. Returns an error if ffmpeg is
|
||||||
|
// not installed or produces no output. The run is bounded by a timeout so a
|
||||||
|
// malformed file cannot hang the caller indefinitely.
|
||||||
|
func extractVideoFrameAt(ctx context.Context, srcPath string, atSec float64) (image.Image, error) {
|
||||||
ctx, cancel := context.WithTimeout(ctx, 30*time.Second)
|
ctx, cancel := context.WithTimeout(ctx, 30*time.Second)
|
||||||
defer cancel()
|
defer cancel()
|
||||||
|
|
||||||
var out bytes.Buffer
|
var out bytes.Buffer
|
||||||
cmd := exec.CommandContext(ctx, "ffmpeg",
|
cmd := exec.CommandContext(ctx, "ffmpeg",
|
||||||
"-ss", "1", // fast input seek; ignored gracefully on short files
|
"-ss", strconv.FormatFloat(atSec, 'f', 3, 64), // fast input seek; ignored gracefully past end
|
||||||
"-i", srcPath,
|
"-i", srcPath,
|
||||||
"-vframes", "1",
|
"-vframes", "1",
|
||||||
"-f", "image2",
|
"-f", "image2",
|
||||||
@@ -274,6 +405,29 @@ func extractVideoFrame(ctx context.Context, srcPath string) (image.Image, error)
|
|||||||
return imaging.Decode(&out)
|
return imaging.Decode(&out)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// videoDurationSeconds returns the container duration in seconds via ffprobe.
|
||||||
|
// Used to seek to the middle of a clip for perceptual hashing and thumbnails.
|
||||||
|
func videoDurationSeconds(ctx context.Context, srcPath string) (float64, error) {
|
||||||
|
ctx, cancel := context.WithTimeout(ctx, 30*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
|
||||||
|
cmd := exec.CommandContext(ctx, "ffprobe",
|
||||||
|
"-v", "error",
|
||||||
|
"-show_entries", "format=duration",
|
||||||
|
"-of", "default=noprint_wrappers=1:nokey=1",
|
||||||
|
srcPath,
|
||||||
|
)
|
||||||
|
out, err := cmd.Output()
|
||||||
|
if err != nil {
|
||||||
|
return 0, fmt.Errorf("ffprobe duration: %w", err)
|
||||||
|
}
|
||||||
|
d, err := strconv.ParseFloat(strings.TrimSpace(string(out)), 64)
|
||||||
|
if err != nil {
|
||||||
|
return 0, fmt.Errorf("ffprobe duration parse %q: %w", out, err)
|
||||||
|
}
|
||||||
|
return d, nil
|
||||||
|
}
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// Path helpers
|
// Path helpers
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|||||||
@@ -0,0 +1,181 @@
|
|||||||
|
package storage
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"image"
|
||||||
|
"image/color"
|
||||||
|
"image/png"
|
||||||
|
"io"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/disintegration/imaging"
|
||||||
|
"github.com/google/uuid"
|
||||||
|
)
|
||||||
|
|
||||||
|
// writeTestImage writes a w×h PNG filled with a distinct (non-placeholder)
|
||||||
|
// colour so a generated thumbnail can be told apart from the grey placeholder.
|
||||||
|
func writeTestImage(t *testing.T, path string, w, h int) {
|
||||||
|
t.Helper()
|
||||||
|
img := image.NewNRGBA(image.Rect(0, 0, w, h))
|
||||||
|
for y := 0; y < h; y++ {
|
||||||
|
for x := 0; x < w; x++ {
|
||||||
|
img.Set(x, y, color.NRGBA{R: 0xC0, G: 0x10, B: 0x20, A: 0xFF})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
f, err := os.Create(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer f.Close()
|
||||||
|
if err := png.Encode(f, img); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDecodeImageLimited(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
p := filepath.Join(dir, "img.png")
|
||||||
|
writeTestImage(t, p, 100, 80) // 8000 px
|
||||||
|
|
||||||
|
if _, err := decodeImageLimited(p, 4000); err == nil {
|
||||||
|
t.Fatal("expected rejection for an image over the pixel cap")
|
||||||
|
}
|
||||||
|
|
||||||
|
img, err := decodeImageLimited(p, 100000)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("expected decode within the cap, got %v", err)
|
||||||
|
}
|
||||||
|
if b := img.Bounds(); b.Dx() != 100 || b.Dy() != 80 {
|
||||||
|
t.Fatalf("unexpected decoded size %v", b.Size())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestThumbnailGeneratesAndCaches exercises the full generation path (semaphore
|
||||||
|
// acquire → decode → fit → encode → cache) and the cache fast path on re-request.
|
||||||
|
func TestThumbnailGeneratesAndCaches(t *testing.T) {
|
||||||
|
files := t.TempDir()
|
||||||
|
thumbs := t.TempDir()
|
||||||
|
id := uuid.New()
|
||||||
|
writeTestImage(t, filepath.Join(files, id.String()), 100, 80)
|
||||||
|
|
||||||
|
s, err := NewDiskStorage(files, thumbs, 160, 160, 1920, 1080, 0, 1)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
rc, err := s.Thumbnail(context.Background(), id)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Thumbnail: %v", err)
|
||||||
|
}
|
||||||
|
data, _ := io.ReadAll(rc)
|
||||||
|
rc.Close()
|
||||||
|
|
||||||
|
out, err := imaging.Decode(bytes.NewReader(data))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("decode thumbnail: %v", err)
|
||||||
|
}
|
||||||
|
// The source fits within 160×160, so it is not upscaled.
|
||||||
|
if b := out.Bounds(); b.Dx() != 100 || b.Dy() != 80 {
|
||||||
|
t.Fatalf("unexpected thumbnail size %v", b.Size())
|
||||||
|
}
|
||||||
|
// Centre pixel should be the source's red, not the grey placeholder.
|
||||||
|
r, g, b, _ := out.At(50, 40).RGBA()
|
||||||
|
if !(r>>8 > g>>8+40 && r>>8 > b>>8+40) {
|
||||||
|
t.Fatalf("thumbnail is not the source image (got r=%d g=%d b=%d) — fell back to placeholder?", r>>8, g>>8, b>>8)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The cache file must now exist, and a second request must serve it.
|
||||||
|
if _, err := os.Stat(s.thumbCachePath(id)); err != nil {
|
||||||
|
t.Fatalf("cache file not written: %v", err)
|
||||||
|
}
|
||||||
|
rc2, err := s.Thumbnail(context.Background(), id)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Thumbnail (cached): %v", err)
|
||||||
|
}
|
||||||
|
rc2.Close()
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestThumbnailFallbackWithoutVips forces the pure-Go pipeline (as if vips were
|
||||||
|
// not installed) and verifies generation still produces the source image.
|
||||||
|
func TestThumbnailFallbackWithoutVips(t *testing.T) {
|
||||||
|
orig := vipsThumbnailPath
|
||||||
|
vipsThumbnailPath = ""
|
||||||
|
t.Cleanup(func() { vipsThumbnailPath = orig })
|
||||||
|
|
||||||
|
files := t.TempDir()
|
||||||
|
thumbs := t.TempDir()
|
||||||
|
id := uuid.New()
|
||||||
|
writeTestImage(t, filepath.Join(files, id.String()), 100, 80)
|
||||||
|
|
||||||
|
s, err := NewDiskStorage(files, thumbs, 160, 160, 1920, 1080, 0, 1)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
rc, err := s.Thumbnail(context.Background(), id)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Thumbnail: %v", err)
|
||||||
|
}
|
||||||
|
data, _ := io.ReadAll(rc)
|
||||||
|
rc.Close()
|
||||||
|
|
||||||
|
out, err := imaging.Decode(bytes.NewReader(data))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("decode thumbnail: %v", err)
|
||||||
|
}
|
||||||
|
if b := out.Bounds(); b.Dx() != 100 || b.Dy() != 80 {
|
||||||
|
t.Fatalf("unexpected thumbnail size %v", b.Size())
|
||||||
|
}
|
||||||
|
r, g, b, _ := out.At(50, 40).RGBA()
|
||||||
|
if !(r>>8 > g>>8+40 && r>>8 > b>>8+40) {
|
||||||
|
t.Fatalf("fallback produced a placeholder, not the source (r=%d g=%d b=%d)", r>>8, g>>8, b>>8)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestPreviewGeneratesAndCaches verifies Preview runs through the same pipeline
|
||||||
|
// with the preview dimensions and its own cache file (not the thumbnail's).
|
||||||
|
func TestPreviewGeneratesAndCaches(t *testing.T) {
|
||||||
|
files := t.TempDir()
|
||||||
|
thumbs := t.TempDir()
|
||||||
|
id := uuid.New()
|
||||||
|
// Larger than the thumbnail box but within the preview box, so the preview
|
||||||
|
// keeps full resolution where a thumbnail would shrink it.
|
||||||
|
writeTestImage(t, filepath.Join(files, id.String()), 400, 300)
|
||||||
|
|
||||||
|
s, err := NewDiskStorage(files, thumbs, 160, 160, 1920, 1080, 0, 1)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
rc, err := s.Preview(context.Background(), id)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Preview: %v", err)
|
||||||
|
}
|
||||||
|
data, _ := io.ReadAll(rc)
|
||||||
|
rc.Close()
|
||||||
|
|
||||||
|
out, err := imaging.Decode(bytes.NewReader(data))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("decode preview: %v", err)
|
||||||
|
}
|
||||||
|
// 400×300 fits within 1920×1080, so the preview is not downscaled.
|
||||||
|
if b := out.Bounds(); b.Dx() != 400 || b.Dy() != 300 {
|
||||||
|
t.Fatalf("unexpected preview size %v", b.Size())
|
||||||
|
}
|
||||||
|
r, g, b, _ := out.At(200, 150).RGBA()
|
||||||
|
if !(r>>8 > g>>8+40 && r>>8 > b>>8+40) {
|
||||||
|
t.Fatalf("preview is not the source image (r=%d g=%d b=%d)", r>>8, g>>8, b>>8)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The preview cache must be written, and the thumbnail cache must not — they
|
||||||
|
// are separate files served by the same code with different dimensions.
|
||||||
|
if _, err := os.Stat(s.previewCachePath(id)); err != nil {
|
||||||
|
t.Fatalf("preview cache not written: %v", err)
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(s.thumbCachePath(id)); err == nil {
|
||||||
|
t.Fatal("thumbnail cache should not exist after a preview-only request")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -55,7 +55,8 @@ CREATE TABLE data.files (
|
|||||||
creator_id smallint NOT NULL REFERENCES core.users(id)
|
creator_id smallint NOT NULL REFERENCES core.users(id)
|
||||||
ON UPDATE CASCADE ON DELETE RESTRICT,
|
ON UPDATE CASCADE ON DELETE RESTRICT,
|
||||||
is_public boolean NOT NULL DEFAULT false,
|
is_public boolean NOT NULL DEFAULT false,
|
||||||
is_deleted boolean NOT NULL DEFAULT false -- soft delete (trash)
|
is_deleted boolean NOT NULL DEFAULT false, -- soft delete (trash)
|
||||||
|
needs_review boolean NOT NULL DEFAULT true -- tagging not yet marked done; cleared explicitly
|
||||||
);
|
);
|
||||||
|
|
||||||
CREATE TABLE data.file_tag (
|
CREATE TABLE data.file_tag (
|
||||||
@@ -75,8 +76,17 @@ CREATE TABLE data.pools (
|
|||||||
creator_id smallint NOT NULL REFERENCES core.users(id)
|
creator_id smallint NOT NULL REFERENCES core.users(id)
|
||||||
ON UPDATE CASCADE ON DELETE RESTRICT,
|
ON UPDATE CASCADE ON DELETE RESTRICT,
|
||||||
is_public boolean NOT NULL DEFAULT false,
|
is_public boolean NOT NULL DEFAULT false,
|
||||||
|
-- File ordering within the pool. 'manual' keeps the user-defined order in
|
||||||
|
-- data.file_pool.position (drag-to-reorder); any other key sorts the pool's
|
||||||
|
-- files automatically by that file field, in which case reordering is disabled.
|
||||||
|
sort_key varchar(32) NOT NULL DEFAULT 'manual',
|
||||||
|
sort_order varchar(4) NOT NULL DEFAULT 'asc',
|
||||||
|
|
||||||
CONSTRAINT uni__pools__name UNIQUE (name)
|
CONSTRAINT uni__pools__name UNIQUE (name),
|
||||||
|
CONSTRAINT chk__pools__sort_key
|
||||||
|
CHECK (sort_key IN ('manual', 'content_datetime', 'created', 'original_name')),
|
||||||
|
CONSTRAINT chk__pools__sort_order
|
||||||
|
CHECK (sort_order IN ('asc', 'desc'))
|
||||||
);
|
);
|
||||||
|
|
||||||
-- `position` uses integer with gaps (e.g. 1000, 2000, 3000) to allow
|
-- `position` uses integer with gaps (e.g. 1000, 2000, 3000) to allow
|
||||||
@@ -91,6 +101,31 @@ CREATE TABLE data.file_pool (
|
|||||||
PRIMARY KEY (file_id, pool_id)
|
PRIMARY KEY (file_id, pool_id)
|
||||||
);
|
);
|
||||||
|
|
||||||
|
-- Precomputed near-duplicate candidates (phash Hamming distance <= threshold),
|
||||||
|
-- (re)built in full by the dedup rescan. Stored once per unordered pair with a
|
||||||
|
-- canonical file_a < file_b ordering so a pair is never duplicated as (a,b)/(b,a).
|
||||||
|
CREATE TABLE data.duplicate_pairs (
|
||||||
|
file_a uuid NOT NULL REFERENCES data.files(id) ON UPDATE CASCADE ON DELETE CASCADE,
|
||||||
|
file_b uuid NOT NULL REFERENCES data.files(id) ON UPDATE CASCADE ON DELETE CASCADE,
|
||||||
|
distance smallint NOT NULL,
|
||||||
|
|
||||||
|
CONSTRAINT chk__duplicate_pairs__order CHECK (file_a < file_b),
|
||||||
|
PRIMARY KEY (file_a, file_b)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- "Not a duplicate" decisions: a global overlay that hides a candidate pair from
|
||||||
|
-- the duplicates view. Survives rescans (the pair may be re-found but stays
|
||||||
|
-- hidden). Same canonical file_a < file_b ordering as data.duplicate_pairs.
|
||||||
|
CREATE TABLE data.duplicate_dismissals (
|
||||||
|
file_a uuid NOT NULL REFERENCES data.files(id) ON UPDATE CASCADE ON DELETE CASCADE,
|
||||||
|
file_b uuid NOT NULL REFERENCES data.files(id) ON UPDATE CASCADE ON DELETE CASCADE,
|
||||||
|
dismissed_by smallint NOT NULL REFERENCES core.users(id) ON UPDATE CASCADE ON DELETE RESTRICT,
|
||||||
|
dismissed_at timestamptz NOT NULL DEFAULT clock_timestamp(),
|
||||||
|
|
||||||
|
CONSTRAINT chk__duplicate_dismissals__order CHECK (file_a < file_b),
|
||||||
|
PRIMARY KEY (file_a, file_b)
|
||||||
|
);
|
||||||
|
|
||||||
COMMENT ON TABLE data.categories IS 'Logical grouping of tags';
|
COMMENT ON TABLE data.categories IS 'Logical grouping of tags';
|
||||||
COMMENT ON TABLE data.tags IS 'File labels/tags';
|
COMMENT ON TABLE data.tags IS 'File labels/tags';
|
||||||
COMMENT ON TABLE data.tag_rules IS 'Auto-tagging rules: when when_tag is assigned, then_tag follows';
|
COMMENT ON TABLE data.tag_rules IS 'Auto-tagging rules: when when_tag is assigned, then_tag follows';
|
||||||
@@ -98,6 +133,8 @@ COMMENT ON TABLE data.files IS 'Managed files; actual content stored on di
|
|||||||
COMMENT ON TABLE data.file_tag IS 'Many-to-many: files <-> tags';
|
COMMENT ON TABLE data.file_tag IS 'Many-to-many: files <-> tags';
|
||||||
COMMENT ON TABLE data.pools IS 'Ordered collections of files';
|
COMMENT ON TABLE data.pools IS 'Ordered collections of files';
|
||||||
COMMENT ON TABLE data.file_pool IS 'Many-to-many: files <-> pools, with ordering';
|
COMMENT ON TABLE data.file_pool IS 'Many-to-many: files <-> pools, with ordering';
|
||||||
|
COMMENT ON TABLE data.duplicate_pairs IS 'Precomputed near-duplicate candidate pairs (perceptual-hash distance)';
|
||||||
|
COMMENT ON TABLE data.duplicate_dismissals IS 'Pairs marked "not a duplicate"; hidden from the duplicates view';
|
||||||
|
|
||||||
COMMENT ON COLUMN data.files.original_name IS 'Original filename at upload time';
|
COMMENT ON COLUMN data.files.original_name IS 'Original filename at upload time';
|
||||||
COMMENT ON COLUMN data.files.content_datetime IS 'Content datetime (e.g. when photo was taken); falls back to EXIF DateTimeOriginal';
|
COMMENT ON COLUMN data.files.content_datetime IS 'Content datetime (e.g. when photo was taken); falls back to EXIF DateTimeOriginal';
|
||||||
@@ -109,6 +146,8 @@ COMMENT ON COLUMN data.file_pool.position IS 'Manual ordering within pool; u
|
|||||||
|
|
||||||
-- +goose Down
|
-- +goose Down
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS data.duplicate_dismissals;
|
||||||
|
DROP TABLE IF EXISTS data.duplicate_pairs;
|
||||||
DROP TABLE IF EXISTS data.file_pool;
|
DROP TABLE IF EXISTS data.file_pool;
|
||||||
DROP TABLE IF EXISTS data.pools;
|
DROP TABLE IF EXISTS data.pools;
|
||||||
DROP TABLE IF EXISTS data.file_tag;
|
DROP TABLE IF EXISTS data.file_tag;
|
||||||
|
|||||||
@@ -20,11 +20,18 @@ CREATE INDEX idx__files__creator_id ON data.files USING hash (creator_id)
|
|||||||
CREATE INDEX idx__files__content_datetime ON data.files USING btree (content_datetime DESC NULLS LAST);
|
CREATE INDEX idx__files__content_datetime ON data.files USING btree (content_datetime DESC NULLS LAST);
|
||||||
CREATE INDEX idx__files__is_deleted ON data.files USING btree (is_deleted) WHERE is_deleted = true;
|
CREATE INDEX idx__files__is_deleted ON data.files USING btree (is_deleted) WHERE is_deleted = true;
|
||||||
CREATE INDEX idx__files__phash ON data.files USING btree (phash) WHERE phash IS NOT NULL;
|
CREATE INDEX idx__files__phash ON data.files USING btree (phash) WHERE phash IS NOT NULL;
|
||||||
|
CREATE INDEX idx__files__needs_review ON data.files USING btree (id) WHERE needs_review = true;
|
||||||
|
|
||||||
-- data.file_tag
|
-- data.file_tag
|
||||||
CREATE INDEX idx__file_tag__tag_id ON data.file_tag USING hash (tag_id);
|
CREATE INDEX idx__file_tag__tag_id ON data.file_tag USING hash (tag_id);
|
||||||
CREATE INDEX idx__file_tag__file_id ON data.file_tag USING hash (file_id);
|
CREATE INDEX idx__file_tag__file_id ON data.file_tag USING hash (file_id);
|
||||||
|
|
||||||
|
-- data.duplicate_pairs / data.duplicate_dismissals
|
||||||
|
-- The composite primary keys cover lookups on file_a; these add the file_b side
|
||||||
|
-- (used by the ON DELETE CASCADE and by the visibility join on the second file).
|
||||||
|
CREATE INDEX idx__duplicate_pairs__file_b ON data.duplicate_pairs USING hash (file_b);
|
||||||
|
CREATE INDEX idx__duplicate_dismissals__file_b ON data.duplicate_dismissals USING hash (file_b);
|
||||||
|
|
||||||
-- data.pools
|
-- data.pools
|
||||||
CREATE INDEX idx__pools__creator_id ON data.pools USING hash (creator_id);
|
CREATE INDEX idx__pools__creator_id ON data.pools USING hash (creator_id);
|
||||||
|
|
||||||
@@ -69,11 +76,14 @@ DROP INDEX IF EXISTS activity.idx__sessions__token_hash;
|
|||||||
DROP INDEX IF EXISTS activity.idx__sessions__user_id;
|
DROP INDEX IF EXISTS activity.idx__sessions__user_id;
|
||||||
DROP INDEX IF EXISTS acl.idx__acl__user;
|
DROP INDEX IF EXISTS acl.idx__acl__user;
|
||||||
DROP INDEX IF EXISTS acl.idx__acl__object;
|
DROP INDEX IF EXISTS acl.idx__acl__object;
|
||||||
|
DROP INDEX IF EXISTS data.idx__duplicate_dismissals__file_b;
|
||||||
|
DROP INDEX IF EXISTS data.idx__duplicate_pairs__file_b;
|
||||||
DROP INDEX IF EXISTS data.idx__file_pool__file_id;
|
DROP INDEX IF EXISTS data.idx__file_pool__file_id;
|
||||||
DROP INDEX IF EXISTS data.idx__file_pool__pool_id;
|
DROP INDEX IF EXISTS data.idx__file_pool__pool_id;
|
||||||
DROP INDEX IF EXISTS data.idx__pools__creator_id;
|
DROP INDEX IF EXISTS data.idx__pools__creator_id;
|
||||||
DROP INDEX IF EXISTS data.idx__file_tag__file_id;
|
DROP INDEX IF EXISTS data.idx__file_tag__file_id;
|
||||||
DROP INDEX IF EXISTS data.idx__file_tag__tag_id;
|
DROP INDEX IF EXISTS data.idx__file_tag__tag_id;
|
||||||
|
DROP INDEX IF EXISTS data.idx__files__needs_review;
|
||||||
DROP INDEX IF EXISTS data.idx__files__phash;
|
DROP INDEX IF EXISTS data.idx__files__phash;
|
||||||
DROP INDEX IF EXISTS data.idx__files__is_deleted;
|
DROP INDEX IF EXISTS data.idx__files__is_deleted;
|
||||||
DROP INDEX IF EXISTS data.idx__files__content_datetime;
|
DROP INDEX IF EXISTS data.idx__files__content_datetime;
|
||||||
|
|||||||
@@ -20,7 +20,8 @@ INSERT INTO activity.action_types (name) VALUES
|
|||||||
('user_login'), ('user_logout'),
|
('user_login'), ('user_logout'),
|
||||||
-- Files
|
-- Files
|
||||||
('file_create'), ('file_edit'), ('file_delete'), ('file_restore'),
|
('file_create'), ('file_edit'), ('file_delete'), ('file_restore'),
|
||||||
('file_permanent_delete'), ('file_replace'),
|
('file_permanent_delete'), ('file_replace'), ('file_review'),
|
||||||
|
('file_merge'), ('duplicate_dismiss'),
|
||||||
-- Tags
|
-- Tags
|
||||||
('tag_create'), ('tag_edit'), ('tag_delete'),
|
('tag_create'), ('tag_edit'), ('tag_delete'),
|
||||||
-- Categories
|
-- Categories
|
||||||
@@ -29,7 +30,7 @@ INSERT INTO activity.action_types (name) VALUES
|
|||||||
('pool_create'), ('pool_edit'), ('pool_delete'),
|
('pool_create'), ('pool_edit'), ('pool_delete'),
|
||||||
-- Relations
|
-- Relations
|
||||||
('file_tag_add'), ('file_tag_remove'),
|
('file_tag_add'), ('file_tag_remove'),
|
||||||
('file_pool_add'), ('file_pool_remove'),
|
('file_pool_add'), ('file_pool_remove'), ('file_pool_reorder'),
|
||||||
-- ACL
|
-- ACL
|
||||||
('acl_change'),
|
('acl_change'),
|
||||||
-- Admin
|
-- Admin
|
||||||
|
|||||||
@@ -0,0 +1,176 @@
|
|||||||
|
# =============================================================================
|
||||||
|
# Tanabata File Manager — Docker Compose
|
||||||
|
#
|
||||||
|
# Quick start:
|
||||||
|
# cp .env.example .env # then edit the secrets
|
||||||
|
# docker compose up -d --build
|
||||||
|
#
|
||||||
|
# Database — two supported modes, selected in .env:
|
||||||
|
#
|
||||||
|
# 1. Bundled Postgres container (default).
|
||||||
|
# COMPOSE_PROFILES=with-db
|
||||||
|
# DATABASE_URL=postgres://tanabata:password@db:5432/tanabata?sslmode=disable
|
||||||
|
#
|
||||||
|
# 2. Postgres already running on the host.
|
||||||
|
# COMPOSE_PROFILES= # empty → the db container is not started
|
||||||
|
# DATABASE_URL=postgres://tanabata:password@host.docker.internal:5432/tanabata?sslmode=disable
|
||||||
|
#
|
||||||
|
# Requires Docker Compose v2.20+ (for depends_on.required).
|
||||||
|
# =============================================================================
|
||||||
|
|
||||||
|
services:
|
||||||
|
app:
|
||||||
|
build:
|
||||||
|
context: .
|
||||||
|
dockerfile: Dockerfile
|
||||||
|
container_name: tfm
|
||||||
|
restart: unless-stopped
|
||||||
|
|
||||||
|
# Give the app time to drain in-flight requests on stop before SIGKILL. Reads
|
||||||
|
# the same SHUTDOWN_TIMEOUT the app uses for its graceful-shutdown deadline
|
||||||
|
# (via env_file below), so the two never drift. Interpolated from .env at
|
||||||
|
# `docker compose` time; defaults to 15s if unset.
|
||||||
|
stop_grace_period: ${SHUTDOWN_TIMEOUT:-15s}
|
||||||
|
|
||||||
|
# All application config (secrets, DATABASE_URL, tunables) comes from .env.
|
||||||
|
env_file: .env
|
||||||
|
|
||||||
|
# Pin STATIC_DIR to the path baked into the image. .env intentionally leaves
|
||||||
|
# it unset; pinning here guarantees in-container SPA serving can't be
|
||||||
|
# disabled by an empty value leaking in through env_file.
|
||||||
|
environment:
|
||||||
|
STATIC_DIR: /app/static
|
||||||
|
|
||||||
|
# Published on loopback only: a reverse proxy on the host (e.g. nginx) fronts
|
||||||
|
# the app and proxies to 127.0.0.1:${APP_PORT}. Binding to 127.0.0.1 keeps the
|
||||||
|
# app off the LAN/WAN — a plain "PORT:42776" would publish on 0.0.0.0 and, since
|
||||||
|
# Docker's DNAT rules sit ahead of the host firewall, bypass ufw/firewalld. The
|
||||||
|
# container always listens on 42776 (Dockerfile default); APP_PORT only changes
|
||||||
|
# the host-published port. Drop the 127.0.0.1 prefix if exposing it directly.
|
||||||
|
ports:
|
||||||
|
- "127.0.0.1:${APP_PORT:-42776}:42776"
|
||||||
|
|
||||||
|
# Two-tier networking. `web` is the app's public-facing bridge (reached via the
|
||||||
|
# published loopback port above; it also provides egress, e.g. to a host
|
||||||
|
# Postgres via host.docker.internal). `backend` is the private tier the app
|
||||||
|
# uses to reach the bundled DB. The DB sits only on `backend`, so nothing on
|
||||||
|
# the host-facing side can reach it.
|
||||||
|
networks:
|
||||||
|
- web
|
||||||
|
- backend
|
||||||
|
|
||||||
|
# Wait for the bundled DB when the with-db profile is active. When using a
|
||||||
|
# host Postgres the db service is disabled, and required:false keeps this
|
||||||
|
# dependency from erroring or auto-starting it.
|
||||||
|
depends_on:
|
||||||
|
db:
|
||||||
|
condition: service_healthy
|
||||||
|
required: false
|
||||||
|
|
||||||
|
# Lets DATABASE_URL reach a Postgres on the host via host.docker.internal
|
||||||
|
# (needed on Linux; harmless elsewhere).
|
||||||
|
extra_hosts:
|
||||||
|
- "host.docker.internal:host-gateway"
|
||||||
|
|
||||||
|
# Run as this uid:gid. Relevant when the mounts below are bind-mounted to
|
||||||
|
# host folders: set PUID/PGID (in .env) to the owner of those folders so the
|
||||||
|
# container can write to them. Defaults to the image's tanabata user
|
||||||
|
# (42776), which owns the named volumes.
|
||||||
|
user: "${PUID:-42776}:${PGID:-42776}"
|
||||||
|
|
||||||
|
# Storage for originals, the thumbnail cache, and the import drop folder.
|
||||||
|
# Each source defaults to a named volume but can be pointed at a specific
|
||||||
|
# host folder via FILES_DIR / THUMBS_DIR / IMPORT_DIR in .env (a path turns
|
||||||
|
# the mount into a host bind mount; a bare name stays a named volume).
|
||||||
|
volumes:
|
||||||
|
- "${FILES_DIR:-app_files}:/data/files"
|
||||||
|
- "${THUMBS_DIR:-app_thumbs}:/data/thumbs"
|
||||||
|
- "${IMPORT_DIR:-app_import}:/data/import"
|
||||||
|
|
||||||
|
db:
|
||||||
|
image: postgres:14-alpine
|
||||||
|
restart: unless-stopped
|
||||||
|
|
||||||
|
# Only started when COMPOSE_PROFILES includes "with-db". Disable it to point
|
||||||
|
# the app at a Postgres running on the host instead.
|
||||||
|
profiles: ["with-db"]
|
||||||
|
|
||||||
|
# Private back-end tier only — never on `web`, never published.
|
||||||
|
networks:
|
||||||
|
- backend
|
||||||
|
|
||||||
|
environment:
|
||||||
|
POSTGRES_DB: ${POSTGRES_DB:-tanabata}
|
||||||
|
POSTGRES_USER: ${POSTGRES_USER:-tanabata}
|
||||||
|
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-password}
|
||||||
|
|
||||||
|
# Defaults to a named volume; set DB_DIR in .env to a host folder to bind
|
||||||
|
# mount it instead. Postgres fixes the folder's ownership itself, so DB_DIR
|
||||||
|
# needs no PUID/PGID.
|
||||||
|
volumes:
|
||||||
|
- "${DB_DIR:-db_data}:/var/lib/postgresql/data"
|
||||||
|
|
||||||
|
# Uncomment to reach the DB from the host (e.g. with psql) for debugging.
|
||||||
|
# ports:
|
||||||
|
# - "5432:5432"
|
||||||
|
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-tanabata} -d ${POSTGRES_DB:-tanabata}"]
|
||||||
|
interval: 5s
|
||||||
|
timeout: 5s
|
||||||
|
retries: 10
|
||||||
|
|
||||||
|
# One-shot maintenance task for duplicate detection: computes missing
|
||||||
|
# perceptual hashes (images + video) and rebuilds the duplicate-pairs table.
|
||||||
|
# It is NOT a daemon — the "tools" profile keeps it out of `docker compose up`;
|
||||||
|
# run it on demand, and it exits when done:
|
||||||
|
#
|
||||||
|
# docker compose run --rm dedup # hashes, then rebuild pairs
|
||||||
|
# docker compose run --rm dedup -pairs # only rebuild pairs (after uploads)
|
||||||
|
# docker compose run --rm dedup -hashes # only backfill hashes
|
||||||
|
#
|
||||||
|
# Reuses the app image, .env, volumes and networks; only the entrypoint differs
|
||||||
|
# (/app/dedup instead of the server). Connects to the same DB the app uses, so
|
||||||
|
# the app's DB (bundled or host) must be reachable when it runs.
|
||||||
|
dedup:
|
||||||
|
build:
|
||||||
|
context: .
|
||||||
|
dockerfile: Dockerfile
|
||||||
|
profiles: ["tools"]
|
||||||
|
env_file: .env
|
||||||
|
networks:
|
||||||
|
- web
|
||||||
|
- backend
|
||||||
|
depends_on:
|
||||||
|
db:
|
||||||
|
condition: service_healthy
|
||||||
|
required: false
|
||||||
|
extra_hosts:
|
||||||
|
- "host.docker.internal:host-gateway"
|
||||||
|
user: "${PUID:-42776}:${PGID:-42776}"
|
||||||
|
volumes:
|
||||||
|
- "${FILES_DIR:-app_files}:/data/files"
|
||||||
|
- "${THUMBS_DIR:-app_thumbs}:/data/thumbs"
|
||||||
|
entrypoint: ["/app/dedup"]
|
||||||
|
restart: "no"
|
||||||
|
|
||||||
|
networks:
|
||||||
|
# Public-facing bridge for this app. The explicit bridge name (instead of
|
||||||
|
# Docker's random br-<hash>) makes it identifiable on the host for tcpdump and
|
||||||
|
# firewall rules.
|
||||||
|
web:
|
||||||
|
driver_opts:
|
||||||
|
com.docker.network.bridge.name: dk-tanabata
|
||||||
|
# Private back-end tier (app ↔ DB). internal:true drops the gateway so the DB
|
||||||
|
# has no route off-host. Note: Linux caps interface names at 15 chars, and
|
||||||
|
# dk-tanabata-bnd is exactly 15 — a longer app name would need a shorter suffix.
|
||||||
|
backend:
|
||||||
|
internal: true
|
||||||
|
driver_opts:
|
||||||
|
com.docker.network.bridge.name: dk-tanabata-bnd
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
app_files:
|
||||||
|
app_thumbs:
|
||||||
|
app_import:
|
||||||
|
db_data:
|
||||||
@@ -0,0 +1,180 @@
|
|||||||
|
# Tanabata File Manager — Architecture
|
||||||
|
|
||||||
|
System-level overview of Tanabata File Manager (TFM). For the product-level
|
||||||
|
requirements see [REQUIREMENTS.md](REQUIREMENTS.md); for per-side detail see
|
||||||
|
[GO_PROJECT_STRUCTURE.md](GO_PROJECT_STRUCTURE.md) (backend) and
|
||||||
|
[FRONTEND_STRUCTURE.md](FRONTEND_STRUCTURE.md) (frontend). The full HTTP contract
|
||||||
|
lives in [`openapi.yaml`](../openapi.yaml).
|
||||||
|
|
||||||
|
## System Context
|
||||||
|
|
||||||
|
TFM is a multi-user, tag-based web file manager for images and video. It is a
|
||||||
|
single deployable unit — one Docker image that serves both the REST API and the
|
||||||
|
built single-page app on one port — plus a PostgreSQL database.
|
||||||
|
|
||||||
|
```
|
||||||
|
┌───────────────────────────────────────────┐
|
||||||
|
Browser / installed │ Reverse proxy (nginx, TLS) │
|
||||||
|
PWA (desktop/mobile) │ host: 443 → 127.0.0.1:${APP_PORT} │
|
||||||
|
│ HTTPS └─────────────────────┬─────────────────────┘
|
||||||
|
└─────────────────────────────────────┼──────────────► 127.0.0.1:42776
|
||||||
|
│
|
||||||
|
┌─────────────────────────▼────────────────────────┐
|
||||||
|
│ Tanabata container (single image) │
|
||||||
|
│ │
|
||||||
|
│ Go server (Gin) │
|
||||||
|
│ ├─ /api/v1/* REST API │
|
||||||
|
│ ├─ /health liveness │
|
||||||
|
│ └─ /* static SPA + index.html │
|
||||||
|
│ fallback │
|
||||||
|
│ │
|
||||||
|
│ Disk: /data/files (originals, name = UUID) │
|
||||||
|
│ /data/thumbs (thumbnail/preview cache) │
|
||||||
|
│ /data/import (server-side import drop) │
|
||||||
|
└─────────────────────────┬────────────────────────┘
|
||||||
|
│ pgx (private network)
|
||||||
|
┌─────────▼─────────┐
|
||||||
|
│ PostgreSQL 14+ │
|
||||||
|
│ (bundled or host)│
|
||||||
|
└───────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
Optional companion process: a one-shot **dedup CLI** (same image, different
|
||||||
|
entrypoint) that backfills perceptual hashes and rebuilds the duplicate-pairs
|
||||||
|
table. It is not a daemon — it is run on demand.
|
||||||
|
|
||||||
|
## Components
|
||||||
|
|
||||||
|
| Component | Tech | Responsibility |
|
||||||
|
| -------------- | -------------------------------------------------------------------------- | -------------------------------------------------------- |
|
||||||
|
| Frontend (SPA) | SvelteKit (adapter-static, `ssr=false`), Svelte 5, Tailwind v4, TypeScript | UI, client routing, PWA/offline, calls the REST API |
|
||||||
|
| API server | Go + Gin, Clean Architecture | REST API, auth, ACL, business logic, thumbnailing, audit |
|
||||||
|
| Database | PostgreSQL 14+ (pgx v5, goose) | All structured data across 4 schemas / 19 tables |
|
||||||
|
| File storage | Local disk, flat, keyed by UUID | Originals + a regenerable thumbnail/preview cache |
|
||||||
|
| dedup CLI | Go (same image) | Offline perceptual-hash backfill + pairs rescan |
|
||||||
|
| Reverse proxy | nginx (host, not shipped) | TLS termination, large-body/streaming config |
|
||||||
|
|
||||||
|
## Backend Architecture (Clean Architecture)
|
||||||
|
|
||||||
|
Dependencies point inward; no layer imports a layer above it.
|
||||||
|
|
||||||
|
```
|
||||||
|
handler → service → port (interfaces) ← db/postgres, storage, imagehash
|
||||||
|
↓
|
||||||
|
domain (entities, value objects, errors) — stdlib only
|
||||||
|
```
|
||||||
|
|
||||||
|
- **domain** — entities and errors, zero internal imports.
|
||||||
|
- **port** — interfaces (repositories, `FileStorage`, `Transactor`).
|
||||||
|
- **service** — use cases; the only place business rules live.
|
||||||
|
- **handler** — Gin HTTP layer; maps domain errors to HTTP status codes.
|
||||||
|
- **db/postgres**, **storage**, **imagehash** — adapters implementing the ports.
|
||||||
|
|
||||||
|
Wiring is manual in `cmd/server/main.go` (no DI framework). See
|
||||||
|
[GO_PROJECT_STRUCTURE.md](GO_PROJECT_STRUCTURE.md) for the file-by-file layout,
|
||||||
|
the transaction/context patterns, and the DI sketch.
|
||||||
|
|
||||||
|
## Request Flow (typical authenticated call)
|
||||||
|
|
||||||
|
1. The SPA sends `Authorization: Bearer <access token>` to `/api/v1/...`.
|
||||||
|
2. Gin middleware runs: security headers → (for `/auth`) per-IP rate limiter →
|
||||||
|
auth middleware validates the JWT and puts `(userID, isAdmin, sessionID)`
|
||||||
|
into the request context.
|
||||||
|
3. The handler parses/validates input and calls a service method
|
||||||
|
(`ctx` first arg).
|
||||||
|
4. The service enforces ACL via `ACLService`, performs the use case — composing
|
||||||
|
repository calls inside a `Transactor.WithTx` when several writes must be
|
||||||
|
atomic — and writes an audit entry.
|
||||||
|
5. Repositories run SQL through pgx (pool or the tx carried in `ctx`).
|
||||||
|
6. The handler serializes the result; domain errors are mapped to
|
||||||
|
`{ code, message, details? }` with the right HTTP status.
|
||||||
|
|
||||||
|
## Cross-Cutting Concerns
|
||||||
|
|
||||||
|
### Authentication & sessions
|
||||||
|
|
||||||
|
JWT bearer auth. A short-lived **access token** (15 min default) authorizes API
|
||||||
|
calls; a long-lived **refresh token** (30 days default) rotates on use and is
|
||||||
|
stored as a hash in `activity.sessions`. A separate **content token** (6 h
|
||||||
|
default) is a single-file capability embedded in media URLs so long video keeps
|
||||||
|
streaming past access-token expiry. The `/auth` endpoints are rate-limited per
|
||||||
|
client IP.
|
||||||
|
|
||||||
|
### Authorization (ACL)
|
||||||
|
|
||||||
|
Private-by-default. Admins see everything; otherwise access requires a `public`
|
||||||
|
flag, creator ownership, or an explicit grant in `acl.permissions` (read / edit).
|
||||||
|
All checks are centralized in `ACLService` and applied before reads and writes.
|
||||||
|
|
||||||
|
### File storage
|
||||||
|
|
||||||
|
Originals are stored flat under `FILES_PATH`, each named by its file UUID (no
|
||||||
|
directory tree, no original-name collisions). Thumbnails and previews are a
|
||||||
|
**regenerable cache** under `THUMBS_CACHE_PATH`: still images via vipsthumbnail
|
||||||
|
(shrink-on-load) with a pure-Go `imaging` fallback, video frames via ffmpeg;
|
||||||
|
metadata/EXIF via exiftool with a pure-Go fallback. Uploads are rejected unless
|
||||||
|
their sniffed MIME type is whitelisted in `core.mime_types`.
|
||||||
|
|
||||||
|
### Near-duplicate detection
|
||||||
|
|
||||||
|
Images are dHash-ed (64-bit perceptual hash) inline on upload; video hashes are
|
||||||
|
backfilled by the dedup CLI. A rescan rebuilds `data.duplicate_pairs` using a
|
||||||
|
BK-tree over Hamming distance (within `DUPLICATE_HASH_THRESHOLD`), and the API
|
||||||
|
groups pairs into connected-component clusters. Dismissed pairs are remembered so
|
||||||
|
they stop resurfacing. See the duplicate sections in
|
||||||
|
[GO_PROJECT_STRUCTURE.md](GO_PROJECT_STRUCTURE.md).
|
||||||
|
|
||||||
|
### Audit logging
|
||||||
|
|
||||||
|
User-visible actions (file/tag/category/pool CRUD, relations, ACL changes,
|
||||||
|
auth, session termination, admin user actions) are recorded in
|
||||||
|
`activity.audit_log` against a seeded set of action types.
|
||||||
|
|
||||||
|
### Frontend / PWA
|
||||||
|
|
||||||
|
Pure client-side SPA: static assets served by the Go binary, with `index.html`
|
||||||
|
as the fallback for client routes. Installable PWA with a service worker for
|
||||||
|
app-shell caching and optional offline viewing of pinned files. See
|
||||||
|
[FRONTEND_STRUCTURE.md](FRONTEND_STRUCTURE.md).
|
||||||
|
|
||||||
|
## Data Model
|
||||||
|
|
||||||
|
PostgreSQL, four schemas (see `backend/migrations/`):
|
||||||
|
|
||||||
|
- **core** — users, MIME whitelist, object types.
|
||||||
|
- **data** — categories, tags, tag rules, files, file–tag, pools, file–pool,
|
||||||
|
duplicate pairs, duplicate dismissals.
|
||||||
|
- **acl** — per-object permission grants.
|
||||||
|
- **activity** — sessions, file/pool views, tag uses, audit log, action types.
|
||||||
|
|
||||||
|
Migrations are goose files embedded via `go:embed` and applied automatically on
|
||||||
|
server startup, so a fresh database bootstraps itself.
|
||||||
|
|
||||||
|
## Deployment
|
||||||
|
|
||||||
|
- **One image, one port.** The multi-stage `Dockerfile` builds the SPA (Node
|
||||||
|
stage) and the static Go binary (Go stage), then ships an Alpine runtime with
|
||||||
|
vips-tools / ffmpeg / exiftool and a non-root user. The server serves both the
|
||||||
|
API and the SPA on port **42776** (the sum of the code points of 七夕).
|
||||||
|
- **Compose.** `docker-compose.yml` runs the app plus, optionally, a bundled
|
||||||
|
PostgreSQL (`with-db` profile); a host Postgres is supported by leaving the
|
||||||
|
profile empty. The app is published on loopback only and expects a host
|
||||||
|
reverse proxy; the DB sits on a private `internal` network with no route
|
||||||
|
off-host. The dedup CLI is a `tools`-profile, run-on-demand service.
|
||||||
|
- **Config.** All runtime config is environment variables, fully documented in
|
||||||
|
[`.env.example`](../.env.example) (1:1 with `config.Config`). Secrets
|
||||||
|
(`JWT_SECRET`, `ADMIN_PASSWORD`, `DATABASE_URL`) are never baked into the image.
|
||||||
|
- **First run.** Migrations auto-apply and the initial admin is bootstrapped
|
||||||
|
from `ADMIN_USERNAME` / `ADMIN_PASSWORD`, so setup is: fill `.env`,
|
||||||
|
`docker compose up`.
|
||||||
|
|
||||||
|
See [DEPLOY.md](DEPLOY.md) for the production deploy (Gitea Actions → host) and
|
||||||
|
the reverse-proxy notes in [README.md](../README.md).
|
||||||
|
|
||||||
|
## Design Constraints & Future Direction
|
||||||
|
|
||||||
|
- **DDD / Clean Architecture** on the server keeps business rules independent of
|
||||||
|
Gin and pgx.
|
||||||
|
- **PostgreSQL-specific adapters are isolated** behind the `port` interfaces (the
|
||||||
|
filter DSL → SQL translation lives in `db/postgres`), leaving room for other
|
||||||
|
database engines in a future version without touching the service layer.
|
||||||
@@ -0,0 +1,185 @@
|
|||||||
|
# Deployment (Gitea Actions → host)
|
||||||
|
|
||||||
|
Tanabata is deployed by a [Gitea Actions](https://docs.gitea.com/usage/actions/overview)
|
||||||
|
workflow ([`.gitea/workflows/deploy.yml`](../.gitea/workflows/deploy.yml)) that
|
||||||
|
runs on the **production host itself**. On every push to `master` it updates the
|
||||||
|
git clone in `/opt/tanabata`, runs the test suite (backend + frontend, in
|
||||||
|
throwaway toolchain containers), and — only if it passes — runs
|
||||||
|
`docker compose up -d --build` there, so the image is built from the
|
||||||
|
freshly-pushed code and the stack is restarted.
|
||||||
|
|
||||||
|
```
|
||||||
|
push master ──> Gitea (container) ──> act_runner (host, "host" label)
|
||||||
|
│ git fetch + reset --hard (in /opt/tanabata)
|
||||||
|
│ run tests (go + node in containers; ephemeral Postgres)
|
||||||
|
└ docker compose up -d --build (only if tests pass)
|
||||||
|
```
|
||||||
|
|
||||||
|
The Gitea server runs in a container, but the **runner runs directly on the host**
|
||||||
|
(shell executor) so it can use the host's git, the host Docker daemon, and the
|
||||||
|
clone in `/opt/tanabata`. Nothing needs a registry — the host builds the image
|
||||||
|
locally. The workflow uses only shell steps, so the host needs just **git** and
|
||||||
|
**docker** (no node, no rsync).
|
||||||
|
|
||||||
|
## What is a runner?
|
||||||
|
|
||||||
|
Gitea (like GitHub) only *coordinates* CI: it stores the workflow, queues jobs,
|
||||||
|
and shows logs. It does **not** execute anything itself. A **runner** is a
|
||||||
|
separate agent program that polls Gitea for queued jobs, runs the steps on a
|
||||||
|
machine you control, and reports results back.
|
||||||
|
|
||||||
|
Gitea's official runner is **act_runner** (a single Go binary; it uses the
|
||||||
|
`act` engine to interpret workflow YAML). One act_runner process can serve many
|
||||||
|
repos. Each runner advertises one or more **labels**, and a job's `runs-on:`
|
||||||
|
picks a runner by label. A label also decides *how* a job runs — the **executor**:
|
||||||
|
|
||||||
|
- **docker executor** — each job runs in a fresh container from an image (e.g.
|
||||||
|
`node:20-bookworm`). Isolated and reproducible; the usual default. Label form
|
||||||
|
at registration: `ubuntu:docker://node:20-bookworm`.
|
||||||
|
- **host / shell executor** — the job runs directly on the host as the runner's
|
||||||
|
user, using host-installed tools. Label form: `host:host`. This is what we use,
|
||||||
|
because the deploy needs the host's Docker daemon and `/opt/tanabata`.
|
||||||
|
|
||||||
|
So `runs-on: host` in the workflow ⇒ "run this job on a runner that registered a
|
||||||
|
`host` label" ⇒ our shell executor on the prod box.
|
||||||
|
|
||||||
|
## One-time setup
|
||||||
|
|
||||||
|
### 1. Enable Actions in Gitea
|
||||||
|
|
||||||
|
Gitea 1.21+ has Actions on by default. Otherwise add to `app.ini` and restart:
|
||||||
|
|
||||||
|
```ini
|
||||||
|
[actions]
|
||||||
|
ENABLED = true
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. A runner user on the host
|
||||||
|
|
||||||
|
Pick (or create) the Linux user the runner runs as. It must be able to use Docker
|
||||||
|
and own the deploy dir — so the workflow needs no `sudo`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo useradd -r -m -d /home/gitea-runner gitea-runner # or reuse an existing user
|
||||||
|
sudo usermod -aG docker gitea-runner # host Docker access
|
||||||
|
```
|
||||||
|
|
||||||
|
The host needs `git` and a Docker engine with the Compose plugin:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo apt install -y git docker.io docker-compose-plugin # Debian/Ubuntu
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Clone the repo to /opt/tanabata once
|
||||||
|
|
||||||
|
The workflow only does `git fetch` + `reset --hard`, so the clone (and its auth)
|
||||||
|
is established here, once. Use a **read-only deploy key** so the host never holds
|
||||||
|
write credentials:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# As the runner user, create a key and add the PUBLIC half to the repo in Gitea:
|
||||||
|
# Repo → Settings → Deploy Keys → Add (read-only)
|
||||||
|
sudo -u gitea-runner ssh-keygen -t ed25519 -f /home/gitea-runner/.ssh/tanabata_deploy -N ''
|
||||||
|
|
||||||
|
# Clone with that key (SSH URL of your Gitea repo):
|
||||||
|
sudo -u gitea-runner GIT_SSH_COMMAND='ssh -i /home/gitea-runner/.ssh/tanabata_deploy' \
|
||||||
|
git clone git@gitea.example.com:you/tanabata.git /opt/tanabata
|
||||||
|
sudo chown -R gitea-runner:gitea-runner /opt/tanabata
|
||||||
|
```
|
||||||
|
|
||||||
|
> HTTPS works too — clone with a URL that carries a read-only token. SSH deploy
|
||||||
|
> keys are the cleaner, per-repo, read-only option.
|
||||||
|
|
||||||
|
After cloning, recurring `git fetch` reuses the remote + key stored in
|
||||||
|
`/opt/tanabata/.git/config`, so the runner itself needs no standing credentials.
|
||||||
|
|
||||||
|
### 4. Register and run act_runner on the host
|
||||||
|
|
||||||
|
Get a registration token in Gitea. **Where you create it sets the runner's
|
||||||
|
scope** (and `--name` is only a display label, unrelated to scope):
|
||||||
|
|
||||||
|
- **Repository** (Tanabata repo → Settings → Actions → Runners) → serves only
|
||||||
|
this repo. **Use this.**
|
||||||
|
- Organization → all repos in the org; Site (admin) → all repos on the instance.
|
||||||
|
|
||||||
|
> Security: this runner is a host/shell executor with access to the Docker
|
||||||
|
> socket — effectively root on the host. Register it at the **repository** level
|
||||||
|
> so only Tanabata's workflows can run on your prod server; a site-wide runner
|
||||||
|
> would let any repo's workflow execute arbitrary commands here.
|
||||||
|
|
||||||
|
Then, as the runner user:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Download act_runner: https://gitea.com/gitea/act_runner/releases
|
||||||
|
act_runner register --no-interactive \
|
||||||
|
--instance https://gitea.example.com \
|
||||||
|
--token <REGISTRATION_TOKEN> \
|
||||||
|
--name prod-host \
|
||||||
|
--labels host:host # <-- maps `runs-on: host` to the shell executor
|
||||||
|
|
||||||
|
# Run it (use a systemd unit in production so it survives reboots):
|
||||||
|
act_runner daemon
|
||||||
|
```
|
||||||
|
|
||||||
|
`--labels host:host` is what makes jobs run **on the host** instead of in a
|
||||||
|
container. The instance URL must be reachable from the host (Gitea's published
|
||||||
|
port / domain — not the in-container address). Registration writes a `.runner`
|
||||||
|
file (the runner's credentials) in the working directory.
|
||||||
|
|
||||||
|
Minimal systemd unit (`/etc/systemd/system/act_runner.service`):
|
||||||
|
|
||||||
|
```ini
|
||||||
|
[Unit]
|
||||||
|
Description=Gitea act_runner
|
||||||
|
After=docker.service
|
||||||
|
Requires=docker.service
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
User=gitea-runner
|
||||||
|
WorkingDirectory=/home/gitea-runner
|
||||||
|
ExecStart=/usr/local/bin/act_runner daemon
|
||||||
|
Restart=always
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target
|
||||||
|
```
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo systemctl enable --now act_runner
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5. Create /opt/tanabata/.env (secrets)
|
||||||
|
|
||||||
|
The workflow **never** writes `.env` — it lives on the host and holds the real
|
||||||
|
secrets and the chosen DB mode. `.env` is git-ignored, so `git reset --hard`
|
||||||
|
leaves it untouched. Create it once:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /opt/tanabata
|
||||||
|
sudo -u gitea-runner cp .env.example .env
|
||||||
|
sudo -u gitea-runner $EDITOR .env # set JWT_SECRET, ADMIN_PASSWORD, DATABASE_URL, etc.
|
||||||
|
```
|
||||||
|
|
||||||
|
See [`.env.example`](../.env.example) for every variable. For the bundled
|
||||||
|
Postgres keep `COMPOSE_PROFILES=with-db`; to use a Postgres already on the host,
|
||||||
|
set it empty and point `DATABASE_URL` at `host.docker.internal`.
|
||||||
|
|
||||||
|
> Data lives in named Docker volumes by default (or the `*_DIR` host paths you
|
||||||
|
> set in `.env`, e.g. `/var/lib/tanabata/...`) — **not** in `/opt/tanabata`. So
|
||||||
|
> `git reset --hard` on the code dir never touches your data.
|
||||||
|
|
||||||
|
## Deploying
|
||||||
|
|
||||||
|
Push to `master` (or hit **Run workflow** on the Actions tab). Watch progress
|
||||||
|
under the repo's **Actions** tab. The first build pulls the Node/Go base images
|
||||||
|
and takes a few minutes; later builds reuse the host's layer cache.
|
||||||
|
|
||||||
|
## Notes / alternatives
|
||||||
|
|
||||||
|
- **Docker-executor runner instead of host.** If you'd rather the runner itself
|
||||||
|
run in a container, register with a Docker label and bind-mount
|
||||||
|
`/var/run/docker.sock` and `/opt/tanabata` into the job (act_runner
|
||||||
|
`config.yaml` → `container.valid_volumes`), then change `runs-on` accordingly.
|
||||||
|
The host executor above is simpler for host deploys.
|
||||||
|
- **Zero-downtime** isn't attempted: `compose up` recreates changed containers.
|
||||||
|
For a single-node setup the brief restart is usually fine.
|
||||||
@@ -1,29 +1,69 @@
|
|||||||
# Tanabata File Manager — Frontend Structure
|
# Tanabata File Manager — Frontend Structure
|
||||||
|
|
||||||
|
> Frontend counterpart of [ARCHITECTURE.md](ARCHITECTURE.md). This document
|
||||||
|
> details the SvelteKit layout, the CSS approach, and the API client.
|
||||||
|
|
||||||
## Stack
|
## Stack
|
||||||
|
|
||||||
- **Framework**: SvelteKit (SPA mode, `ssr: false`)
|
- **Framework**: SvelteKit in **SPA mode** (`adapter-static`, `ssr = false`),
|
||||||
- **Language**: TypeScript
|
Svelte 5 (runes)
|
||||||
- **CSS**: Tailwind CSS + CSS custom properties (hybrid)
|
- **Language**: TypeScript (strict)
|
||||||
- **API types**: Auto-generated via openapi-typescript
|
- **Build**: Vite 7
|
||||||
- **PWA**: Service worker + web manifest
|
- **CSS**: Tailwind CSS **v4** via `@tailwindcss/vite` + CSS custom properties
|
||||||
|
(`@theme` in `app.css`) — no `tailwind.config.*` / `postcss.config.*` file
|
||||||
|
- **API types**: auto-generated via `openapi-typescript` (`src/lib/api/schema.ts`)
|
||||||
|
- **PWA**: service worker + web manifest
|
||||||
- **Font**: Epilogue (variable weight)
|
- **Font**: Epilogue (variable weight)
|
||||||
|
- **Dev**: `vite-mock-plugin.ts` serves a mock API so the UI can run without the
|
||||||
|
Go backend
|
||||||
- **Package manager**: npm
|
- **Package manager**: npm
|
||||||
|
|
||||||
|
## SPA mode — why SvelteKit without the server
|
||||||
|
|
||||||
|
This frontend runs as a **pure client-side SPA**: `adapter-static` with
|
||||||
|
`fallback: 'index.html'` and `ssr = false` globally (see
|
||||||
|
`src/routes/+layout.ts`). There is no Node server in production — the build is
|
||||||
|
static assets, and the only backend is the Go API. SvelteKit is used here
|
||||||
|
purely as an SPA framework: file-based routing, the client router, and build
|
||||||
|
tooling.
|
||||||
|
|
||||||
|
**SvelteKit features we _do_ use:**
|
||||||
|
|
||||||
|
- File-based routing with nested layouts (`admin/` has its own guard) and
|
||||||
|
dynamic segments (`[id]`).
|
||||||
|
- The client router: `goto`, the `page` store/state, `afterNavigate`,
|
||||||
|
`navigating`.
|
||||||
|
- **Shallow routing** — `pushState`/`replaceState` + `page.state`. The
|
||||||
|
Immich-style file viewer in `files/` and `pools/[id]/` opens as an overlay
|
||||||
|
over the still-mounted list via shallow routing, so the browser back button
|
||||||
|
dismisses it without reloading the grid. This is the single biggest reason we
|
||||||
|
stay on SvelteKit rather than a plain router.
|
||||||
|
- `load` functions, used _only_ as client-side route guards (auth redirect,
|
||||||
|
admin redirect, `/` → `/files`).
|
||||||
|
- `$lib` alias, generated `./$types`, Vite/HMR integration.
|
||||||
|
|
||||||
|
**SvelteKit features we deliberately do _not_ use** (the "server half"):
|
||||||
|
|
||||||
|
- SSR / hydration.
|
||||||
|
- `+page.server.ts`, `+server.ts` endpoints, form actions — all data goes
|
||||||
|
through the Go API via the `$lib/api` client.
|
||||||
|
- `hooks.*`, prerendering, server-only modules — no `hooks.server.ts` /
|
||||||
|
`hooks.client.ts` files exist.
|
||||||
|
|
||||||
|
**Decision: stay on SvelteKit, do not migrate to a bare Svelte + router SPA.**
|
||||||
|
The project already _is_ an SPA, so there is no runtime gain from switching
|
||||||
|
(adapter-static tree-shakes the unused server bits; the client-runtime size
|
||||||
|
difference is negligible). A migration would mean re-implementing nested
|
||||||
|
layouts, guards, dynamic params, and — most painfully — shallow routing /
|
||||||
|
history-state overlays by hand, for zero benefit. New contributors should not
|
||||||
|
expect SSR, endpoints, or hooks to do anything here; that is intentional.
|
||||||
|
|
||||||
## Monorepo Layout
|
## Monorepo Layout
|
||||||
|
|
||||||
```
|
```
|
||||||
tanabata/
|
tanabata/
|
||||||
├── backend/ ← Go project (go.mod in here)
|
├── backend/ ← Go project (go.mod in here)
|
||||||
│ ├── cmd/
|
|
||||||
│ ├── internal/
|
|
||||||
│ ├── migrations/
|
|
||||||
│ ├── go.mod
|
|
||||||
│ └── go.sum
|
|
||||||
│
|
|
||||||
├── frontend/ ← SvelteKit project (package.json in here)
|
├── frontend/ ← SvelteKit project (package.json in here)
|
||||||
│ └── (see below)
|
|
||||||
│
|
|
||||||
├── openapi.yaml ← Shared API contract (root level)
|
├── openapi.yaml ← Shared API contract (root level)
|
||||||
├── docker-compose.yml
|
├── docker-compose.yml
|
||||||
├── Dockerfile
|
├── Dockerfile
|
||||||
@@ -31,353 +71,206 @@ tanabata/
|
|||||||
└── README.md
|
└── README.md
|
||||||
```
|
```
|
||||||
|
|
||||||
`openapi.yaml` lives at repository root — both backend and frontend
|
`openapi.yaml` lives at repository root — both backend and frontend reference
|
||||||
reference it. The frontend generates types from it; the backend
|
it. The frontend generates types from it; the backend implements it.
|
||||||
validates its handlers against it.
|
|
||||||
|
|
||||||
## Frontend Directory Layout
|
## Frontend Directory Layout
|
||||||
|
|
||||||
```
|
```
|
||||||
frontend/
|
frontend/
|
||||||
├── package.json
|
├── package.json
|
||||||
├── svelte.config.js
|
├── svelte.config.js # adapter-static, fallback: index.html
|
||||||
├── vite.config.ts
|
├── vite.config.ts # plugins: tailwindcss(), sveltekit(), mockApiPlugin()
|
||||||
|
├── vite-mock-plugin.ts # dev-only mock API (run the UI without the Go backend)
|
||||||
├── tsconfig.json
|
├── tsconfig.json
|
||||||
├── tailwind.config.ts
|
|
||||||
├── postcss.config.js
|
|
||||||
│
|
│
|
||||||
├── src/
|
├── static/ # Copied verbatim into the build
|
||||||
│ ├── app.html # Shell HTML (PWA meta, font preload)
|
│ ├── manifest.webmanifest # PWA manifest
|
||||||
│ ├── app.css # Tailwind directives + CSS custom properties
|
│ ├── browserconfig.xml
|
||||||
│ ├── hooks.server.ts # Server hooks (not used in SPA mode)
|
│ ├── robots.txt
|
||||||
│ ├── hooks.client.ts # Client hooks (global error handling)
|
│ ├── favicon.ico
|
||||||
│ │
|
│ ├── fonts/
|
||||||
│ ├── lib/ # Shared code ($lib/ alias)
|
│ │ └── Epilogue-VariableFont_wght.ttf
|
||||||
│ │ │
|
│ └── images/ # PWA icons, section icons (svg), login decorations
|
||||||
│ │ ├── api/ # API client layer
|
|
||||||
│ │ │ ├── client.ts # Base fetch wrapper: auth headers, token refresh,
|
|
||||||
│ │ │ │ # error parsing, base URL
|
|
||||||
│ │ │ ├── files.ts # listFiles, getFile, uploadFile, deleteFile, etc.
|
|
||||||
│ │ │ ├── tags.ts # listTags, createTag, getTag, updateTag, etc.
|
|
||||||
│ │ │ ├── categories.ts # Category API functions
|
|
||||||
│ │ │ ├── pools.ts # Pool API functions
|
|
||||||
│ │ │ ├── auth.ts # login, logout, refresh, listSessions
|
|
||||||
│ │ │ ├── acl.ts # getPermissions, setPermissions
|
|
||||||
│ │ │ ├── users.ts # getMe, updateMe, admin user CRUD
|
|
||||||
│ │ │ ├── audit.ts # queryAuditLog
|
|
||||||
│ │ │ ├── schema.ts # AUTO-GENERATED from openapi.yaml (do not edit)
|
|
||||||
│ │ │ └── types.ts # Friendly type aliases:
|
|
||||||
│ │ │ # export type File = components["schemas"]["File"]
|
|
||||||
│ │ │ # export type Tag = components["schemas"]["Tag"]
|
|
||||||
│ │ │
|
|
||||||
│ │ ├── components/ # Reusable UI components
|
|
||||||
│ │ │ │
|
|
||||||
│ │ │ ├── layout/ # App shell
|
|
||||||
│ │ │ │ ├── Navbar.svelte # Bottom navigation bar (mobile-first)
|
|
||||||
│ │ │ │ ├── Header.svelte # Section header with sorting controls
|
|
||||||
│ │ │ │ ├── SelectionBar.svelte # Floating bar for multi-select actions
|
|
||||||
│ │ │ │ └── Loader.svelte # Full-screen loading overlay
|
|
||||||
│ │ │ │
|
|
||||||
│ │ │ ├── file/ # File-related components
|
|
||||||
│ │ │ │ ├── FileGrid.svelte # Thumbnail grid with infinite scroll
|
|
||||||
│ │ │ │ ├── FileCard.svelte # Single thumbnail (160×160, selectable)
|
|
||||||
│ │ │ │ ├── FileViewer.svelte # Full-screen preview with prev/next navigation
|
|
||||||
│ │ │ │ ├── FileUpload.svelte # Upload form + drag-and-drop zone
|
|
||||||
│ │ │ │ ├── FileDetail.svelte # Metadata editor (notes, datetime, tags)
|
|
||||||
│ │ │ │ └── FilterBar.svelte # DSL filter builder UI
|
|
||||||
│ │ │ │
|
|
||||||
│ │ │ ├── tag/ # Tag-related components
|
|
||||||
│ │ │ │ ├── TagBadge.svelte # Colored pill with tag name
|
|
||||||
│ │ │ │ ├── TagPicker.svelte # Searchable tag selector (add/remove)
|
|
||||||
│ │ │ │ ├── TagList.svelte # Tag grid for section view
|
|
||||||
│ │ │ │ └── TagRuleEditor.svelte # Auto-tag rule management
|
|
||||||
│ │ │ │
|
|
||||||
│ │ │ ├── pool/ # Pool-related components
|
|
||||||
│ │ │ │ ├── PoolCard.svelte # Pool preview card
|
|
||||||
│ │ │ │ ├── PoolFileList.svelte # Ordered file list with drag reorder
|
|
||||||
│ │ │ │ └── PoolDetail.svelte # Pool metadata editor
|
|
||||||
│ │ │ │
|
|
||||||
│ │ │ ├── acl/ # Access control components
|
|
||||||
│ │ │ │ └── PermissionEditor.svelte # User permission grid
|
|
||||||
│ │ │ │
|
|
||||||
│ │ │ └── common/ # Shared primitives
|
|
||||||
│ │ │ ├── Button.svelte
|
|
||||||
│ │ │ ├── Modal.svelte
|
|
||||||
│ │ │ ├── ConfirmDialog.svelte
|
|
||||||
│ │ │ ├── Toast.svelte
|
|
||||||
│ │ │ ├── InfiniteScroll.svelte
|
|
||||||
│ │ │ ├── Pagination.svelte
|
|
||||||
│ │ │ ├── SortDropdown.svelte
|
|
||||||
│ │ │ ├── SearchInput.svelte
|
|
||||||
│ │ │ ├── ColorPicker.svelte
|
|
||||||
│ │ │ ├── Checkbox.svelte # Three-state: checked, unchecked, partial
|
|
||||||
│ │ │ └── EmptyState.svelte
|
|
||||||
│ │ │
|
|
||||||
│ │ ├── stores/ # Svelte stores (global state)
|
|
||||||
│ │ │ ├── auth.ts # Current user, JWT tokens, isAuthenticated
|
|
||||||
│ │ │ ├── selection.ts # Selected item IDs, selection mode toggle
|
|
||||||
│ │ │ ├── sorting.ts # Per-section sort key + order (persisted to localStorage)
|
|
||||||
│ │ │ ├── theme.ts # Dark/light mode (persisted, respects prefers-color-scheme)
|
|
||||||
│ │ │ └── toast.ts # Notification queue (success, error, info)
|
|
||||||
│ │ │
|
|
||||||
│ │ └── utils/ # Pure helper functions
|
|
||||||
│ │ ├── format.ts # formatDate, formatFileSize, formatDuration
|
|
||||||
│ │ ├── dsl.ts # Filter DSL builder: UI state → query string
|
|
||||||
│ │ ├── pwa.ts # PWA reset, cache clear, update prompt
|
|
||||||
│ │ └── keyboard.ts # Keyboard shortcut helpers (Ctrl+A, Escape, etc.)
|
|
||||||
│ │
|
|
||||||
│ ├── routes/ # SvelteKit file-based routing
|
|
||||||
│ │ │
|
|
||||||
│ │ ├── +layout.svelte # Root layout: Navbar, theme wrapper, toast container
|
|
||||||
│ │ ├── +layout.ts # Root load: auth guard → redirect to /login if no token
|
|
||||||
│ │ │
|
|
||||||
│ │ ├── +page.svelte # / → redirect to /files
|
|
||||||
│ │ │
|
|
||||||
│ │ ├── login/
|
|
||||||
│ │ │ └── +page.svelte # Login form (decorative Tanabata images)
|
|
||||||
│ │ │
|
|
||||||
│ │ ├── files/
|
|
||||||
│ │ │ ├── +page.svelte # File grid: filter bar, sort, multi-select, upload
|
|
||||||
│ │ │ ├── +page.ts # Load: initial file list (cursor page)
|
|
||||||
│ │ │ ├── [id]/
|
|
||||||
│ │ │ │ ├── +page.svelte # File view: preview, metadata, tags, ACL
|
|
||||||
│ │ │ │ └── +page.ts # Load: file detail + tags
|
|
||||||
│ │ │ └── trash/
|
|
||||||
│ │ │ ├── +page.svelte # Trash: restore / permanent delete
|
|
||||||
│ │ │ └── +page.ts
|
|
||||||
│ │ │
|
|
||||||
│ │ ├── tags/
|
|
||||||
│ │ │ ├── +page.svelte # Tag list: search, sort, multi-select
|
|
||||||
│ │ │ ├── +page.ts
|
|
||||||
│ │ │ ├── new/
|
|
||||||
│ │ │ │ └── +page.svelte # Create tag form
|
|
||||||
│ │ │ └── [id]/
|
|
||||||
│ │ │ ├── +page.svelte # Tag detail: edit, category, rules, parent tags
|
|
||||||
│ │ │ └── +page.ts
|
|
||||||
│ │ │
|
|
||||||
│ │ ├── categories/
|
|
||||||
│ │ │ ├── +page.svelte # Category list
|
|
||||||
│ │ │ ├── +page.ts
|
|
||||||
│ │ │ ├── new/
|
|
||||||
│ │ │ │ └── +page.svelte
|
|
||||||
│ │ │ └── [id]/
|
|
||||||
│ │ │ ├── +page.svelte # Category detail: edit, view tags
|
|
||||||
│ │ │ └── +page.ts
|
|
||||||
│ │ │
|
|
||||||
│ │ ├── pools/
|
|
||||||
│ │ │ ├── +page.svelte # Pool list
|
|
||||||
│ │ │ ├── +page.ts
|
|
||||||
│ │ │ ├── new/
|
|
||||||
│ │ │ │ └── +page.svelte
|
|
||||||
│ │ │ └── [id]/
|
|
||||||
│ │ │ ├── +page.svelte # Pool detail: files (reorderable), filter, edit
|
|
||||||
│ │ │ └── +page.ts
|
|
||||||
│ │ │
|
|
||||||
│ │ ├── settings/
|
|
||||||
│ │ │ ├── +page.svelte # Profile: name, password, active sessions
|
|
||||||
│ │ │ └── +page.ts
|
|
||||||
│ │ │
|
|
||||||
│ │ └── admin/
|
|
||||||
│ │ ├── +layout.svelte # Admin layout: restrict to is_admin
|
|
||||||
│ │ ├── users/
|
|
||||||
│ │ │ ├── +page.svelte # User management list
|
|
||||||
│ │ │ ├── +page.ts
|
|
||||||
│ │ │ └── [id]/
|
|
||||||
│ │ │ ├── +page.svelte # User detail: role, block/unblock
|
|
||||||
│ │ │ └── +page.ts
|
|
||||||
│ │ └── audit/
|
|
||||||
│ │ ├── +page.svelte # Audit log with filters
|
|
||||||
│ │ └── +page.ts
|
|
||||||
│ │
|
|
||||||
│ └── service-worker.ts # PWA: offline cache for pinned files, app shell caching
|
|
||||||
│
|
│
|
||||||
└── static/
|
└── src/
|
||||||
├── favicon.png
|
├── app.html # Shell HTML (PWA meta, font preload)
|
||||||
├── favicon.ico
|
├── app.css # `@import 'tailwindcss'` + `@theme` custom properties
|
||||||
├── manifest.webmanifest # PWA manifest (name, icons, theme_color)
|
├── app.d.ts # Ambient types
|
||||||
├── images/
|
├── service-worker.ts # PWA: app-shell + pinned-file offline cache
|
||||||
│ ├── tanabata-left.png # Login page decorations (from current design)
|
│
|
||||||
│ ├── tanabata-right.png
|
├── lib/ # Shared code ($lib alias)
|
||||||
│ └── icons/ # PWA icons (192×192, 512×512, etc.)
|
│ ├── index.ts
|
||||||
└── fonts/
|
│ │
|
||||||
└── Epilogue-VariableFont_wght.ttf
|
│ ├── api/ # API client layer
|
||||||
|
│ │ ├── client.ts # fetch wrapper: bearer auth, 401 refresh+retry, error parsing,
|
||||||
|
│ │ │ # upload-with-progress (XHR), NDJSON streaming; exports `api`
|
||||||
|
│ │ ├── auth.ts # login, refresh, logout, sessions
|
||||||
|
│ │ ├── tags.ts # tag + tag-rule calls
|
||||||
|
│ │ ├── categories.ts # category calls
|
||||||
|
│ │ ├── duplicates.ts # duplicate list / dismiss / resolve
|
||||||
|
│ │ ├── schema.ts # AUTO-GENERATED from openapi.yaml (gitignored; do not edit)
|
||||||
|
│ │ └── types.ts # Friendly aliases: components['schemas'][...]
|
||||||
|
│ │
|
||||||
|
│ ├── components/
|
||||||
|
│ │ ├── layout/
|
||||||
|
│ │ │ ├── Header.svelte # Section header with sorting controls
|
||||||
|
│ │ │ ├── SelectionBar.svelte # Floating bar for multi-select actions
|
||||||
|
│ │ │ └── KeyboardHelp.svelte # Keyboard-shortcut overlay
|
||||||
|
│ │ │
|
||||||
|
│ │ ├── file/
|
||||||
|
│ │ │ ├── FileCard.svelte # Single thumbnail (160×160, selectable)
|
||||||
|
│ │ │ ├── Thumb.svelte # Lazy-loaded thumbnail (IntersectionObserver)
|
||||||
|
│ │ │ ├── FileViewer.svelte # Full-screen viewer with prev/next
|
||||||
|
│ │ │ ├── FileUpload.svelte # Upload form + drag-and-drop
|
||||||
|
│ │ │ ├── FilterBar.svelte # DSL filter builder UI
|
||||||
|
│ │ │ ├── MetadataEditor.svelte # Notes / datetime / metadata (nested) editor
|
||||||
|
│ │ │ ├── TagPicker.svelte # Searchable tag selector (add/remove)
|
||||||
|
│ │ │ ├── PoolPicker.svelte # Add-to-pool dialog
|
||||||
|
│ │ │ ├── BulkTagEditor.svelte # Multi-select tag add/remove
|
||||||
|
│ │ │ └── DuplicateMergeDialog.svelte # Field-by-field duplicate resolution
|
||||||
|
│ │ │
|
||||||
|
│ │ ├── tag/
|
||||||
|
│ │ │ ├── TagBadge.svelte # Colored pill
|
||||||
|
│ │ │ └── TagRuleEditor.svelte # Auto-tag rule management
|
||||||
|
│ │ │
|
||||||
|
│ │ └── common/
|
||||||
|
│ │ ├── ConfirmDialog.svelte
|
||||||
|
│ │ └── InfiniteScroll.svelte # Below-the-fold lazy loading on scroll
|
||||||
|
│ │
|
||||||
|
│ ├── stores/ # Svelte stores (global state)
|
||||||
|
│ │ ├── auth.ts # Current user + JWT tokens (persisted, cross-tab sync)
|
||||||
|
│ │ ├── selection.ts # Selected item IDs, selection mode
|
||||||
|
│ │ ├── sorting.ts # Per-section sort key + order (persisted)
|
||||||
|
│ │ ├── theme.ts # Dark/light theme (persisted)
|
||||||
|
│ │ ├── appSettings.ts # Misc client-side settings
|
||||||
|
│ │ ├── listScroll.ts # Restore list scroll position after overlay/back
|
||||||
|
│ │ └── sectionCache.ts # Cached list snapshots, invalidated on mutation
|
||||||
|
│ │
|
||||||
|
│ └── utils/ # Pure helpers
|
||||||
|
│ ├── dsl.ts # Filter DSL builder: UI state → query string
|
||||||
|
│ ├── metadata.ts # Nested metadata <-> editor rows
|
||||||
|
│ ├── pwa.ts # PWA reset / update prompt
|
||||||
|
│ └── rovingGrid.svelte.ts # Roving-tabindex keyboard grid navigation
|
||||||
|
│
|
||||||
|
└── routes/ # SvelteKit file-based routing (guards only in load)
|
||||||
|
├── +layout.svelte # Root layout: nav, theme
|
||||||
|
├── +layout.ts # ssr=false; root auth guard
|
||||||
|
├── +page.svelte / +page.ts # / → redirect to /files
|
||||||
|
├── login/+page.svelte
|
||||||
|
├── files/
|
||||||
|
│ ├── +page.svelte / +page.ts # Grid: filter, sort, multi-select, upload
|
||||||
|
│ ├── [id]/+page.svelte # File view (also opened as shallow-routing overlay)
|
||||||
|
│ ├── duplicates/+page.svelte # Duplicate clusters
|
||||||
|
│ └── trash/+page.svelte # Trash: restore / permanent delete
|
||||||
|
├── tags/ { +page.svelte, new/, [id]/ }
|
||||||
|
├── categories/ { +page.svelte, new/, [id]/ }
|
||||||
|
├── pools/ { +page.svelte, new/, [id]/ }
|
||||||
|
├── settings/+page.svelte # Profile: name, password, sessions, import path
|
||||||
|
└── admin/
|
||||||
|
├── +layout.svelte / +layout.ts # Restrict to admins
|
||||||
|
├── users/{ +page.svelte, [id]/ }
|
||||||
|
└── audit/+page.svelte
|
||||||
```
|
```
|
||||||
|
|
||||||
## Key Architecture Decisions
|
## Key Architecture Decisions
|
||||||
|
|
||||||
### CSS Hybrid: Tailwind + Custom Properties
|
### CSS: Tailwind v4 + Custom Properties
|
||||||
|
|
||||||
Theme colors defined as CSS custom properties in `app.css`:
|
Tailwind v4 is configured **in CSS**, not in a JS config file. `app.css` imports
|
||||||
|
Tailwind and declares the theme tokens as CSS custom properties inside `@theme`;
|
||||||
|
Tailwind then generates utilities (`bg-bg-primary`, `text-text-primary`,
|
||||||
|
`font-sans`, …) from those tokens automatically.
|
||||||
|
|
||||||
```css
|
```css
|
||||||
@tailwind base;
|
/* src/app.css */
|
||||||
@tailwind components;
|
@import "tailwindcss";
|
||||||
@tailwind utilities;
|
|
||||||
|
|
||||||
:root {
|
@theme {
|
||||||
--color-bg-primary: #312F45;
|
--color-bg-primary: #312f45;
|
||||||
--color-bg-secondary: #181721;
|
--color-bg-secondary: #181721;
|
||||||
--color-bg-elevated: #111118;
|
--color-bg-elevated: #111118;
|
||||||
--color-accent: #9592B5;
|
--color-accent: #9592b5;
|
||||||
--color-accent-hover: #7D7AA4;
|
--color-accent-hover: #7d7aa4;
|
||||||
--color-text-primary: #f0f0f0;
|
--color-text-primary: #f0f0f0;
|
||||||
--color-text-muted: #9999AD;
|
--color-tag-default: #444455;
|
||||||
--color-danger: #DB6060;
|
/* … info / danger / warning / success / nav tokens … */
|
||||||
--color-info: #4DC7ED;
|
|
||||||
--color-warning: #F5E872;
|
|
||||||
--color-tag-default: #444455;
|
|
||||||
}
|
|
||||||
|
|
||||||
:root[data-theme="light"] {
|
--font-sans: "Epilogue", sans-serif;
|
||||||
--color-bg-primary: #f5f5f5;
|
|
||||||
--color-bg-secondary: #ffffff;
|
|
||||||
/* ... */
|
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
Tailwind references them in `tailwind.config.ts`:
|
Dark theme is primary; the light theme overrides the same custom properties.
|
||||||
|
|
||||||
```ts
|
|
||||||
export default {
|
|
||||||
theme: {
|
|
||||||
extend: {
|
|
||||||
colors: {
|
|
||||||
bg: {
|
|
||||||
primary: 'var(--color-bg-primary)',
|
|
||||||
secondary: 'var(--color-bg-secondary)',
|
|
||||||
elevated: 'var(--color-bg-elevated)',
|
|
||||||
},
|
|
||||||
accent: {
|
|
||||||
DEFAULT: 'var(--color-accent)',
|
|
||||||
hover: 'var(--color-accent-hover)',
|
|
||||||
},
|
|
||||||
// ...
|
|
||||||
},
|
|
||||||
fontFamily: {
|
|
||||||
sans: ['Epilogue', 'sans-serif'],
|
|
||||||
},
|
|
||||||
},
|
|
||||||
},
|
|
||||||
darkMode: 'class', // controlled via data-theme attribute
|
|
||||||
};
|
|
||||||
```
|
|
||||||
|
|
||||||
Usage in components: `<div class="bg-bg-primary text-text-primary rounded-xl p-4">`.
|
Usage in components: `<div class="bg-bg-primary text-text-primary rounded-xl p-4">`.
|
||||||
Complex cases use scoped `<style>` inside `.svelte` files.
|
Complex cases use scoped `<style>` inside `.svelte` files.
|
||||||
|
|
||||||
### API Client Pattern
|
### API Client Pattern
|
||||||
|
|
||||||
`client.ts` — thin wrapper around fetch:
|
`src/lib/api/client.ts` is a thin fetch wrapper exporting a generic `api`
|
||||||
|
object. It attaches the bearer token, transparently handles a single `401`
|
||||||
|
refresh-and-retry (deduplicating concurrent refreshes and syncing rotated
|
||||||
|
tokens across tabs), parses `{ code, message, details }` errors into `ApiError`,
|
||||||
|
and invalidates cached list snapshots on mutation. It also provides
|
||||||
|
`uploadWithProgress` (XHR, for upload progress) and `postStream` (NDJSON, for
|
||||||
|
the live import progress).
|
||||||
|
|
||||||
```ts
|
```ts
|
||||||
// $lib/api/client.ts
|
// $lib/api/client.ts (shape)
|
||||||
import { authStore } from '$lib/stores/auth';
|
|
||||||
|
|
||||||
const BASE = '/api/v1';
|
|
||||||
|
|
||||||
async function request<T>(path: string, init?: RequestInit): Promise<T> {
|
|
||||||
const token = get(authStore).accessToken;
|
|
||||||
const res = await fetch(BASE + path, {
|
|
||||||
...init,
|
|
||||||
headers: {
|
|
||||||
'Content-Type': 'application/json',
|
|
||||||
...(token && { Authorization: `Bearer ${token}` }),
|
|
||||||
...init?.headers,
|
|
||||||
},
|
|
||||||
});
|
|
||||||
if (res.status === 401) {
|
|
||||||
// attempt refresh, retry once
|
|
||||||
}
|
|
||||||
if (!res.ok) {
|
|
||||||
const err = await res.json();
|
|
||||||
throw new ApiError(res.status, err.code, err.message, err.details);
|
|
||||||
}
|
|
||||||
if (res.status === 204) return undefined as T;
|
|
||||||
return res.json();
|
|
||||||
}
|
|
||||||
|
|
||||||
export const api = {
|
export const api = {
|
||||||
get: <T>(path: string) => request<T>(path),
|
get: <T>(path) => request<T>(path),
|
||||||
post: <T>(path: string, body?: unknown) =>
|
post: <T>(path, body?) =>
|
||||||
request<T>(path, { method: 'POST', body: JSON.stringify(body) }),
|
request<T>(path, { method: "POST", body: JSON.stringify(body) }),
|
||||||
patch: <T>(path: string, body?: unknown) =>
|
patch: <T>(path, body?) =>
|
||||||
request<T>(path, { method: 'PATCH', body: JSON.stringify(body) }),
|
request<T>(path, { method: "PATCH", body: JSON.stringify(body) }),
|
||||||
put: <T>(path: string, body?: unknown) =>
|
put: <T>(path, body?) =>
|
||||||
request<T>(path, { method: 'PUT', body: JSON.stringify(body) }),
|
request<T>(path, { method: "PUT", body: JSON.stringify(body) }),
|
||||||
delete: <T>(path: string) => request<T>(path, { method: 'DELETE' }),
|
delete: <T>(path) => request<T>(path, { method: "DELETE" }),
|
||||||
upload: <T>(path: string, formData: FormData) =>
|
upload: <T>(path, fd) => request<T>(path, { method: "POST", body: fd }),
|
||||||
request<T>(path, { method: 'POST', body: formData, headers: {} }),
|
|
||||||
};
|
};
|
||||||
```
|
```
|
||||||
|
|
||||||
Domain-specific modules use it:
|
Resource modules (`auth.ts`, `tags.ts`, `categories.ts`, `duplicates.ts`) wrap
|
||||||
|
`api` with typed helpers. Endpoints without a dedicated module (files, pools,
|
||||||
```ts
|
users, acl, audit) are called through `api.*` directly from their route
|
||||||
// $lib/api/files.ts
|
components.
|
||||||
import { api } from './client';
|
|
||||||
import type { File, FileCursorPage } from './types';
|
|
||||||
|
|
||||||
export function listFiles(params: Record<string, string>) {
|
|
||||||
const qs = new URLSearchParams(params).toString();
|
|
||||||
return api.get<FileCursorPage>(`/files?${qs}`);
|
|
||||||
}
|
|
||||||
|
|
||||||
export function uploadFile(formData: FormData) {
|
|
||||||
return api.upload<File>('/files', formData);
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Type Generation
|
### Type Generation
|
||||||
|
|
||||||
Script in `package.json`:
|
|
||||||
|
|
||||||
```json
|
```json
|
||||||
|
// package.json
|
||||||
{
|
{
|
||||||
"scripts": {
|
"scripts": {
|
||||||
"generate:types": "openapi-typescript ../openapi.yaml -o src/lib/api/schema.ts",
|
"generate:types": "openapi-typescript ../openapi.yaml -o src/lib/api/schema.ts",
|
||||||
"dev": "npm run generate:types && vite dev",
|
"dev": "npm run generate:types && vite dev",
|
||||||
"build": "npm run generate:types && vite build"
|
"build": "npm run generate:types && vite build"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
Friendly aliases in `types.ts`:
|
`schema.ts` is generated (and gitignored) — never edit it by hand. `types.ts`
|
||||||
|
re-exports friendly aliases:
|
||||||
|
|
||||||
```ts
|
```ts
|
||||||
import type { components } from './schema';
|
import type { components } from "./schema";
|
||||||
|
export type File = components["schemas"]["File"];
|
||||||
export type File = components['schemas']['File'];
|
export type Tag = components["schemas"]["Tag"];
|
||||||
export type Tag = components['schemas']['Tag'];
|
// …
|
||||||
export type Category = components['schemas']['Category'];
|
|
||||||
export type Pool = components['schemas']['Pool'];
|
|
||||||
export type FileCursorPage = components['schemas']['FileCursorPage'];
|
|
||||||
export type TagOffsetPage = components['schemas']['TagOffsetPage'];
|
|
||||||
export type Error = components['schemas']['Error'];
|
|
||||||
// ...
|
|
||||||
```
|
```
|
||||||
|
|
||||||
### SPA Mode
|
### SPA Mode (build)
|
||||||
|
|
||||||
`svelte.config.js`:
|
|
||||||
|
|
||||||
```js
|
```js
|
||||||
import adapter from '@sveltejs/adapter-static';
|
// svelte.config.js
|
||||||
|
import adapter from "@sveltejs/adapter-static";
|
||||||
export default {
|
export default { kit: { adapter: adapter({ fallback: "index.html" }) } };
|
||||||
kit: {
|
|
||||||
adapter: adapter({ fallback: 'index.html' }),
|
|
||||||
// SPA: all routes handled client-side
|
|
||||||
},
|
|
||||||
};
|
|
||||||
```
|
```
|
||||||
|
|
||||||
The Go backend serves `index.html` for all non-API routes (SPA fallback).
|
The Go backend serves `index.html` for all non-API routes (SPA fallback, see
|
||||||
In development, Vite dev server proxies `/api` to the Go backend.
|
`handler/static.go`). In development the Vite dev server serves the UI and the
|
||||||
|
mock plugin (or a proxied Go backend) answers `/api`.
|
||||||
|
|
||||||
### PWA
|
### PWA
|
||||||
|
|
||||||
`service-worker.ts` handles:
|
`service-worker.ts` handles app-shell caching (HTML/CSS/JS/fonts) and optional
|
||||||
- App shell caching (HTML, CSS, JS, fonts)
|
user-pinned file caching for offline viewing; `utils/pwa.ts` exposes the reset /
|
||||||
- User-pinned file caching (explicit, via UI button)
|
update flow (clear caches and reload from the server, keeping pinned files).
|
||||||
- Cache versioning and cleanup on update
|
|
||||||
- Reset function (clear all caches except pinned files)
|
|
||||||
|
|||||||
@@ -1,18 +1,28 @@
|
|||||||
# Tanabata File Manager — Go Project Structure
|
# Tanabata File Manager — Go Project Structure
|
||||||
|
|
||||||
|
> Backend counterpart of [ARCHITECTURE.md](ARCHITECTURE.md). This document
|
||||||
|
> details the Go layout, the layer rules, and the key backend decisions.
|
||||||
|
|
||||||
## Stack
|
## Stack
|
||||||
|
|
||||||
- **Router**: Gin
|
- **Router**: Gin
|
||||||
- **Database**: pgx v5 (pgxpool)
|
- **Database**: pgx v5 (pgxpool)
|
||||||
- **Migrations**: goose v3 + go:embed (auto-migrate on startup)
|
- **Migrations**: goose v3 + `go:embed` (auto-applied on startup)
|
||||||
- **Auth**: JWT (golang-jwt/jwt/v5)
|
- **Auth**: JWT (golang-jwt/jwt/v5), Bearer access tokens + rotating refresh tokens
|
||||||
- **Config**: environment variables via .env (joho/godotenv)
|
- **Config**: environment variables via `.env` (joho/godotenv)
|
||||||
- **Logging**: slog (stdlib, Go 1.21+)
|
- **Logging**: slog (stdlib)
|
||||||
- **Validation**: go-playground/validator/v10
|
- **Metadata**: exiftool (external, preferred) with a pure-Go EXIF fallback
|
||||||
- **EXIF**: rwcarlsen/goexif or dsoprea/go-exif
|
(rwcarlsen/goexif)
|
||||||
- **Image processing**: disintegration/imaging (thumbnails, previews)
|
- **Thumbnails / previews**: vipsthumbnail (external, shrink-on-load) and ffmpeg
|
||||||
|
(video frames), with a pure-Go fallback (disintegration/imaging)
|
||||||
|
- **Near-duplicate detection**: 64-bit dHash perceptual hashing + a BK-tree /
|
||||||
|
Hamming-distance pairing (`internal/imagehash`, `internal/service/duplicate_*`)
|
||||||
- **Architecture**: Clean Architecture (domain → service → repository/handler)
|
- **Architecture**: Clean Architecture (domain → service → repository/handler)
|
||||||
|
|
||||||
|
The binary is fully static (`CGO_ENABLED=0`). External tools are invoked as
|
||||||
|
subprocesses when present and are optional — the pure-Go paths keep the server
|
||||||
|
working without them.
|
||||||
|
|
||||||
## Monorepo Layout
|
## Monorepo Layout
|
||||||
|
|
||||||
```
|
```
|
||||||
@@ -31,81 +41,98 @@ tanabata/
|
|||||||
```
|
```
|
||||||
backend/
|
backend/
|
||||||
├── cmd/
|
├── cmd/
|
||||||
│ └── server/
|
│ ├── server/
|
||||||
│ └── main.go # Entrypoint: config → DB → migrate → wire → run
|
│ │ └── main.go # Entrypoint: config → DB → migrate → bootstrap admin → wire → serve
|
||||||
|
│ └── dedup/
|
||||||
|
│ └── main.go # Offline maintenance CLI: perceptual-hash backfill + duplicate-pairs rescan
|
||||||
│
|
│
|
||||||
├── internal/
|
├── internal/
|
||||||
│ │
|
│ │
|
||||||
│ ├── domain/ # Pure business entities & value objects
|
│ ├── domain/ # Pure business entities & value objects (stdlib only)
|
||||||
│ │ ├── file.go # File, FileFilter, FilePage
|
│ │ ├── file.go # File, FileFilter, FileListParams, FilePage
|
||||||
│ │ ├── tag.go # Tag, TagRule
|
│ │ ├── tag.go # Tag, TagRule
|
||||||
│ │ ├── category.go # Category
|
│ │ ├── category.go # Category
|
||||||
│ │ ├── pool.go # Pool, PoolFile
|
│ │ ├── pool.go # Pool, PoolFile
|
||||||
│ │ ├── user.go # User, Session
|
│ │ ├── user.go # User, Session
|
||||||
│ │ ├── acl.go # Permission, ObjectType
|
│ │ ├── acl.go # Permission, ObjectType
|
||||||
│ │ ├── audit.go # AuditEntry, ActionType
|
│ │ ├── audit.go # AuditEntry, ActionType
|
||||||
│ │ └── errors.go # Domain error types (ErrNotFound, ErrForbidden, etc.)
|
│ │ ├── duplicate.go # DuplicatePair, PHashEntry
|
||||||
|
│ │ ├── context.go # WithUser / UserFromContext (identity + session in ctx)
|
||||||
|
│ │ └── errors.go # Domain error types (ErrNotFound, ErrForbidden, …)
|
||||||
│ │
|
│ │
|
||||||
│ ├── port/ # Interfaces (ports) — contracts between layers
|
│ ├── port/ # Interfaces (ports) — contracts between layers
|
||||||
│ │ ├── repository.go # FileRepo, TagRepo, CategoryRepo, PoolRepo,
|
│ │ ├── repository.go # Transactor, FileRepo, TagRepo, TagRuleRepo, CategoryRepo,
|
||||||
│ │ │ # UserRepo, SessionRepo, ACLRepo, AuditRepo,
|
│ │ │ # PoolRepo, UserRepo, SessionRepo, ACLRepo, AuditRepo,
|
||||||
│ │ │ # MimeRepo, TagRuleRepo
|
│ │ │ # MimeRepo, DuplicatePairRepo, DismissalRepo
|
||||||
│ │ └── storage.go # FileStorage interface (disk operations)
|
│ │ └── storage.go # FileStorage (originals + thumbnail/preview cache)
|
||||||
│ │
|
│ │
|
||||||
│ ├── service/ # Business logic (use cases)
|
│ ├── service/ # Business logic (use cases)
|
||||||
│ │ ├── file_service.go # Upload, update, delete, trash/restore, replace,
|
│ │ ├── file_service.go # Upload, update, delete, trash/restore, replace, import, filter/list
|
||||||
│ │ │ # import, filter/list, duplicate detection
|
│ │ ├── tag_service.go # CRUD + auto-tag (rule) application
|
||||||
│ │ ├── tag_service.go # CRUD + auto-tag application logic
|
│ │ ├── category_service.go # CRUD (thin: repo + ACL + audit)
|
||||||
│ │ ├── category_service.go # CRUD (thin, delegates to repo + ACL + audit)
|
|
||||||
│ │ ├── pool_service.go # CRUD + file ordering, add/remove files
|
│ │ ├── pool_service.go # CRUD + file ordering, add/remove files
|
||||||
│ │ ├── auth_service.go # Login, logout, JWT issue/refresh, session management
|
│ │ ├── auth_service.go # Login, logout, JWT issue/refresh, content tokens, sessions
|
||||||
│ │ ├── acl_service.go # Permission checks, grant/revoke
|
│ │ ├── acl_service.go # Permission checks, grant/revoke
|
||||||
│ │ ├── audit_service.go # Log actions, query audit log
|
│ │ ├── audit_service.go # Log actions, query audit log
|
||||||
│ │ └── user_service.go # Profile update, admin CRUD, block/unblock
|
│ │ ├── user_service.go # Profile update, admin CRUD, block/unblock, EnsureAdmin
|
||||||
|
│ │ ├── duplicate_service.go # Cluster / resolve (merge) / dismiss + rescan orchestration
|
||||||
|
│ │ ├── duplicate_index.go # BK-tree, Hamming pairing, connected-component clustering
|
||||||
|
│ │ └── metadata.go # EXIF / media metadata extraction (exiftool + pure-Go fallback)
|
||||||
│ │
|
│ │
|
||||||
│ ├── handler/ # HTTP layer (Gin handlers)
|
│ ├── handler/ # HTTP layer (Gin handlers)
|
||||||
│ │ ├── router.go # Route registration, middleware wiring
|
│ │ ├── router.go # Route registration, middleware, security headers, SPA fallback
|
||||||
│ │ ├── middleware.go # Auth middleware (JWT extraction → context)
|
│ │ ├── middleware.go # Auth middleware (JWT / content token → context)
|
||||||
│ │ ├── request.go # Common request parsing helpers
|
│ │ ├── ratelimit.go # Per-IP token-bucket limiter for /auth
|
||||||
│ │ ├── response.go # Error/success response builders
|
│ │ ├── response.go # Error/success builders, domain-error → HTTP mapping
|
||||||
|
│ │ ├── static.go # Built SPA serving + index.html fallback
|
||||||
│ │ ├── file_handler.go # /files endpoints
|
│ │ ├── file_handler.go # /files endpoints
|
||||||
│ │ ├── tag_handler.go # /tags endpoints
|
│ │ ├── duplicate_handler.go # /files/duplicates endpoints
|
||||||
|
│ │ ├── tag_handler.go # /tags endpoints (+ file–tag relations)
|
||||||
│ │ ├── category_handler.go # /categories endpoints
|
│ │ ├── category_handler.go # /categories endpoints
|
||||||
│ │ ├── pool_handler.go # /pools endpoints
|
│ │ ├── pool_handler.go # /pools endpoints
|
||||||
│ │ ├── auth_handler.go # /auth endpoints
|
│ │ ├── auth_handler.go # /auth endpoints
|
||||||
│ │ ├── acl_handler.go # /acl endpoints
|
│ │ ├── acl_handler.go # /acl endpoints
|
||||||
│ │ ├── user_handler.go # /users endpoints
|
│ │ ├── user_handler.go # /users endpoints
|
||||||
│ │ └── audit_handler.go # /audit endpoints
|
│ │ └── audit_handler.go # /audit endpoint
|
||||||
│ │
|
│ │
|
||||||
│ ├── db/ # Database adapters
|
│ ├── db/ # Database adapters
|
||||||
│ │ ├── db.go # Common helpers: pagination, repo factory, transactor base
|
│ │ ├── db.go # Shared helpers: Querier, tx-from-context, ScanRow, limit/offset clamps
|
||||||
│ │ └── postgres/ # PostgreSQL implementation
|
│ │ └── postgres/ # PostgreSQL implementation
|
||||||
│ │ ├── postgres.go # pgxpool init, tx-from-context helpers
|
│ │ ├── postgres.go # pgxpool init, Transactor, conn-or-tx helper
|
||||||
│ │ ├── file_repo.go # FileRepo implementation
|
│ │ ├── file_repo.go # FileRepo (incl. perceptual-hash projections)
|
||||||
│ │ ├── tag_repo.go # TagRepo + TagRuleRepo implementation
|
│ │ ├── tag_repo.go # TagRepo + TagRuleRepo
|
||||||
│ │ ├── category_repo.go # CategoryRepo implementation
|
│ │ ├── category_repo.go # CategoryRepo
|
||||||
│ │ ├── pool_repo.go # PoolRepo implementation
|
│ │ ├── pool_repo.go # PoolRepo
|
||||||
│ │ ├── user_repo.go # UserRepo implementation
|
│ │ ├── user_repo.go # UserRepo
|
||||||
│ │ ├── session_repo.go # SessionRepo implementation
|
│ │ ├── session_repo.go # SessionRepo
|
||||||
│ │ ├── acl_repo.go # ACLRepo implementation
|
│ │ ├── acl_repo.go # ACLRepo
|
||||||
│ │ ├── audit_repo.go # AuditRepo implementation
|
│ │ ├── audit_repo.go # AuditRepo
|
||||||
│ │ ├── mime_repo.go # MimeRepo implementation
|
│ │ ├── mime_repo.go # MimeRepo
|
||||||
│ │ └── filter_parser.go # DSL → SQL WHERE clause builder
|
│ │ ├── duplicate_repo.go # DuplicatePairRepo + DismissalRepo
|
||||||
|
│ │ └── filter_parser.go # Filter DSL → SQL WHERE clause builder
|
||||||
│ │
|
│ │
|
||||||
│ ├── storage/ # File storage adapter
|
│ ├── storage/ # File storage adapter
|
||||||
│ │ └── disk.go # FileStorage implementation (read/write/delete on disk)
|
│ │ └── disk.go # FileStorage on disk: originals + thumbnail/preview cache
|
||||||
|
│ │ # (vipsthumbnail / ffmpeg / pure-Go imaging)
|
||||||
|
│ │
|
||||||
|
│ ├── imagehash/ # Perceptual hashing (64-bit dHash) for near-duplicate detection
|
||||||
|
│ │ └── imagehash.go
|
||||||
|
│ │
|
||||||
|
│ ├── integration/ # End-to-end HTTP tests against a disposable Postgres
|
||||||
|
│ │ └── server_test.go
|
||||||
│ │
|
│ │
|
||||||
│ └── config/ # Configuration
|
│ └── config/ # Configuration
|
||||||
│ └── config.go # Struct + loader from env vars
|
│ └── config.go # Config struct + loader from env vars
|
||||||
│
|
│
|
||||||
├── migrations/ # SQL migration files (goose format)
|
├── migrations/ # SQL migration files (goose format), embedded via go:embed
|
||||||
│ ├── 001_init_schemas.sql
|
│ ├── 001_init_schemas.sql
|
||||||
│ ├── 002_core_tables.sql
|
│ ├── 002_core_tables.sql
|
||||||
│ ├── 003_data_tables.sql
|
│ ├── 003_data_tables.sql
|
||||||
│ ├── 004_acl_tables.sql
|
│ ├── 004_acl_tables.sql
|
||||||
│ ├── 005_activity_tables.sql
|
│ ├── 005_activity_tables.sql
|
||||||
│ ├── 006_indexes.sql
|
│ ├── 006_indexes.sql
|
||||||
│ └── 007_seed_data.sql
|
│ ├── 007_seed_data.sql
|
||||||
|
│ └── embed.go # //go:embed *.sql → migrations.FS
|
||||||
│
|
│
|
||||||
├── go.mod
|
├── go.mod
|
||||||
└── go.sum
|
└── go.sum
|
||||||
@@ -126,6 +153,7 @@ handler → service → port (interfaces) ← db/postgres / storage
|
|||||||
- **db/postgres/**: imports domain/, port/, and db/ (common helpers). Implements port interfaces.
|
- **db/postgres/**: imports domain/, port/, and db/ (common helpers). Implements port interfaces.
|
||||||
- **db/**: imports domain/ and port/. Shared utilities for all DB adapters.
|
- **db/**: imports domain/ and port/. Shared utilities for all DB adapters.
|
||||||
- **storage/**: imports domain/ and port/. Implements FileStorage.
|
- **storage/**: imports domain/ and port/. Implements FileStorage.
|
||||||
|
- **imagehash/**: leaf package (stdlib + image libs); used by service/ and storage/.
|
||||||
|
|
||||||
No layer may import a layer above it. No circular dependencies.
|
No layer may import a layer above it. No circular dependencies.
|
||||||
|
|
||||||
@@ -133,138 +161,126 @@ No layer may import a layer above it. No circular dependencies.
|
|||||||
|
|
||||||
### Dependency Injection (Wiring)
|
### Dependency Injection (Wiring)
|
||||||
|
|
||||||
Manual wiring in `cmd/server/main.go`. No DI frameworks.
|
Manual wiring in `cmd/server/main.go`. No DI frameworks. Constructors take their
|
||||||
|
collaborators explicitly; the shape below matches the real signatures.
|
||||||
|
|
||||||
```go
|
```go
|
||||||
// Pseudocode
|
// Pseudocode — see cmd/server/main.go for the exact calls.
|
||||||
pool := postgres.NewPool(cfg.DatabaseURL)
|
pool := postgres.NewPool(ctx, cfg.DatabaseURL)
|
||||||
goose.Up(pool, migrations)
|
goose.Up(stdlib.OpenDBFromPool(pool), ".") // migrations.FS embedded
|
||||||
|
|
||||||
// Repos (all from internal/db/postgres/)
|
|
||||||
fileRepo := postgres.NewFileRepo(pool)
|
|
||||||
tagRepo := postgres.NewTagRepo(pool)
|
|
||||||
// ...
|
|
||||||
|
|
||||||
// Storage
|
// Storage
|
||||||
diskStore := storage.NewDiskStorage(cfg.FilesPath)
|
diskStorage := storage.NewDiskStorage(
|
||||||
|
cfg.FilesPath, cfg.ThumbsCachePath,
|
||||||
|
cfg.ThumbWidth, cfg.ThumbHeight, cfg.PreviewWidth, cfg.PreviewHeight,
|
||||||
|
cfg.ThumbMaxPixels, cfg.ThumbConcurrency,
|
||||||
|
)
|
||||||
|
|
||||||
|
// Repos (all from internal/db/postgres/)
|
||||||
|
fileRepo := postgres.NewFileRepo(pool)
|
||||||
|
// … tag, tagRule, category, pool, user, session, acl, audit, mime,
|
||||||
|
// duplicatePair, dismissal repos + transactor
|
||||||
|
|
||||||
// Services
|
// Services
|
||||||
aclSvc := service.NewACLService(aclRepo, objectTypeRepo)
|
authSvc := service.NewAuthService(userRepo, sessionRepo,
|
||||||
auditSvc := service.NewAuditService(auditRepo, actionTypeRepo)
|
cfg.JWTSecret, cfg.JWTAccessTTL, cfg.JWTRefreshTTL, cfg.ContentTokenTTL)
|
||||||
fileSvc := service.NewFileService(fileRepo, mimeRepo, tagRepo, diskStore, aclSvc, auditSvc)
|
aclSvc := service.NewACLService(aclRepo, fileRepo, tagRepo, categoryRepo, poolRepo, transactor)
|
||||||
tagSvc := service.NewTagService(tagRepo, tagRuleRepo, aclSvc, auditSvc)
|
auditSvc := service.NewAuditService(auditRepo)
|
||||||
// ...
|
tagSvc := service.NewTagService(tagRepo, tagRuleRepo, aclSvc, auditSvc, transactor)
|
||||||
|
dupSvc := service.NewDuplicateService(fileRepo, duplicatePairRepo, dismissalRepo,
|
||||||
|
aclSvc, auditSvc, transactor, cfg.DuplicateHashThreshold)
|
||||||
|
fileSvc := service.NewFileService(fileRepo, mimeRepo, diskStorage,
|
||||||
|
aclSvc, auditSvc, tagSvc, transactor, cfg.ImportPath)
|
||||||
|
// … category, pool, user services
|
||||||
|
|
||||||
// Handlers
|
// Bootstrap the initial admin from env (idempotent).
|
||||||
fileHandler := handler.NewFileHandler(fileSvc, tagSvc)
|
userSvc.EnsureAdmin(ctx, cfg.AdminUsername, cfg.AdminPassword)
|
||||||
// ...
|
|
||||||
|
|
||||||
router := handler.NewRouter(cfg, fileHandler, tagHandler, ...)
|
// Handlers → router (also wires trusted proxies + optional static SPA dir)
|
||||||
router.Run(cfg.ListenAddr)
|
router, _ := handler.NewRouter(authMiddleware, authHandler, fileHandler,
|
||||||
|
duplicateHandler, tagHandler, categoryHandler, poolHandler,
|
||||||
|
userHandler, aclHandler, auditHandler, cfg.StaticDir, cfg.TrustedProxies)
|
||||||
|
srv.ListenAndServe()
|
||||||
```
|
```
|
||||||
|
|
||||||
### Context Propagation
|
### Context Propagation
|
||||||
|
|
||||||
Every service method receives `context.Context` as the first argument.
|
Every service method receives `context.Context` as the first argument.
|
||||||
The handler extracts user info from JWT (via middleware) and puts it
|
The auth middleware parses the JWT and puts the caller's identity (user id,
|
||||||
into context. Services read the current user from context for ACL checks
|
admin flag, session id) into the context. Services read it for ACL checks and
|
||||||
and audit logging.
|
audit logging.
|
||||||
|
|
||||||
```go
|
```go
|
||||||
// middleware.go
|
// handler/middleware.go
|
||||||
func (m *AuthMiddleware) Handle(c *gin.Context) {
|
claims := parseJWT(c.GetHeader("Authorization"))
|
||||||
claims := parseJWT(c.GetHeader("Authorization"))
|
ctx := domain.WithUser(c.Request.Context(), claims.UserID, claims.IsAdmin, claims.SessionID)
|
||||||
ctx := domain.WithUser(c.Request.Context(), claims.UserID, claims.IsAdmin)
|
c.Request = c.Request.WithContext(ctx)
|
||||||
c.Request = c.Request.WithContext(ctx)
|
|
||||||
c.Next()
|
|
||||||
}
|
|
||||||
|
|
||||||
// domain/context.go
|
// domain/context.go
|
||||||
type ctxKey int
|
func WithUser(ctx context.Context, userID int16, isAdmin bool, sessionID int) context.Context
|
||||||
const userKey ctxKey = iota
|
func UserFromContext(ctx context.Context) (userID int16, isAdmin bool, sessionID int)
|
||||||
func WithUser(ctx context.Context, userID int16, isAdmin bool) context.Context { ... }
|
|
||||||
func UserFromContext(ctx context.Context) (userID int16, isAdmin bool) { ... }
|
|
||||||
```
|
```
|
||||||
|
|
||||||
### Transaction Management
|
### Transaction Management
|
||||||
|
|
||||||
Repository interfaces include a `Transactor`:
|
The `Transactor` port lets services compose multiple repo calls atomically.
|
||||||
|
The postgres implementation stores the active `pgx.Tx` in the context; repo
|
||||||
|
methods pick it up via a conn-or-tx helper, so the same method works inside or
|
||||||
|
outside a transaction.
|
||||||
|
|
||||||
```go
|
```go
|
||||||
// port/repository.go
|
// port/repository.go
|
||||||
type Transactor interface {
|
type Transactor interface {
|
||||||
WithTx(ctx context.Context, fn func(ctx context.Context) error) error
|
WithTx(ctx context.Context, fn func(ctx context.Context) error) error
|
||||||
}
|
}
|
||||||
```
|
|
||||||
|
|
||||||
The postgres implementation wraps `pgxpool.Pool.BeginTx`. Inside `fn`,
|
// service/file_service.go (sketch)
|
||||||
all repo calls use the transaction from context. This allows services
|
func (s *FileService) Upload(ctx context.Context, p UploadParams) (*domain.File, error) {
|
||||||
to compose multiple repo calls in a single transaction:
|
|
||||||
|
|
||||||
```go
|
|
||||||
// service/file_service.go
|
|
||||||
func (s *FileService) Upload(ctx context.Context, input UploadInput) (*domain.File, error) {
|
|
||||||
return s.tx.WithTx(ctx, func(ctx context.Context) error {
|
return s.tx.WithTx(ctx, func(ctx context.Context) error {
|
||||||
file, err := s.fileRepo.Create(ctx, ...) // uses tx
|
created, err := s.files.Create(ctx, f) // uses tx from ctx
|
||||||
if err != nil { return err }
|
// apply initial tags, etc., in the same tx
|
||||||
for _, tagID := range input.TagIDs {
|
return err
|
||||||
s.tagRepo.AddFileTag(ctx, file.ID, tagID) // same tx
|
|
||||||
}
|
|
||||||
s.auditRepo.Log(ctx, ...) // same tx
|
|
||||||
return nil
|
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
### ACL Check Pattern
|
### ACL Check Pattern
|
||||||
|
|
||||||
ACL logic is centralized in `ACLService`. Other services call it before
|
ACL logic is centralized in `ACLService`. Other services call it before any
|
||||||
any data mutation or retrieval:
|
mutation or retrieval. The model is private-by-default: admins see everything;
|
||||||
|
otherwise a `public` flag, creator ownership, or an explicit `acl.permissions`
|
||||||
|
grant is required.
|
||||||
|
|
||||||
```go
|
```go
|
||||||
// service/acl_service.go
|
// service/acl_service.go (shape)
|
||||||
func (s *ACLService) CanView(ctx context.Context, objectType string, objectID uuid.UUID) error {
|
func (s *ACLService) CanView(ctx context.Context, userID int16, isAdmin bool,
|
||||||
userID, isAdmin := domain.UserFromContext(ctx)
|
creatorID int16, isPublic bool, objectType int16, objectID uuid.UUID) (bool, error)
|
||||||
if isAdmin { return nil }
|
func (s *ACLService) CanEdit(ctx context.Context, userID int16, isAdmin bool,
|
||||||
// Check is_public on the object
|
creatorID int16, objectType int16, objectID uuid.UUID) (bool, error)
|
||||||
// If not public, check creator_id == userID
|
|
||||||
// If not creator, check acl.permissions
|
|
||||||
// Return domain.ErrForbidden if none match
|
|
||||||
}
|
|
||||||
```
|
```
|
||||||
|
|
||||||
### Error Mapping
|
### Error Mapping
|
||||||
|
|
||||||
Domain errors → HTTP status codes (handled in handler/response.go):
|
Domain errors → HTTP status codes (handled in handler/response.go):
|
||||||
|
|
||||||
| Domain Error | HTTP Status | Error Code |
|
| Domain Error | HTTP Status | Error Code |
|
||||||
|-----------------------|-------------|-------------------|
|
| ------------------ | ----------- | ---------------- |
|
||||||
| ErrNotFound | 404 | not_found |
|
| ErrNotFound | 404 | not_found |
|
||||||
| ErrForbidden | 403 | forbidden |
|
| ErrForbidden | 403 | forbidden |
|
||||||
| ErrUnauthorized | 401 | unauthorized |
|
| ErrUnauthorized | 401 | unauthorized |
|
||||||
| ErrConflict | 409 | conflict |
|
| ErrConflict | 409 | conflict |
|
||||||
| ErrValidation | 400 | validation_error |
|
| ErrValidation | 400 | validation_error |
|
||||||
| ErrUnsupportedMIME | 415 | unsupported_mime |
|
| ErrUnsupportedMIME | 415 | unsupported_mime |
|
||||||
| (unexpected) | 500 | internal_error |
|
| (unexpected) | 500 | internal_error |
|
||||||
|
|
||||||
### Filter DSL
|
### Filter DSL
|
||||||
|
|
||||||
The DSL parser lives in `db/postgres/filter_parser.go` because it produces
|
The DSL parser lives in `db/postgres/filter_parser.go` because it produces SQL
|
||||||
SQL WHERE clauses — it is a PostgreSQL-specific adapter concern.
|
WHERE clauses — a PostgreSQL-specific adapter concern. The service layer passes
|
||||||
The service layer passes the raw DSL string to the repository; the
|
the raw DSL string down; the repository parses it and builds the query. For a
|
||||||
repository parses it and builds the query.
|
different DBMS, a corresponding parser would live in `db/<dbms>/filter_parser.go`.
|
||||||
|
|
||||||
For a different DBMS, a corresponding parser would live in
|
|
||||||
`db/<dbms>/filter_parser.go`.
|
|
||||||
|
|
||||||
The interface:
|
|
||||||
```go
|
```go
|
||||||
// port/repository.go
|
|
||||||
type FileRepo interface {
|
|
||||||
List(ctx context.Context, params FileListParams) (*domain.FilePage, error)
|
|
||||||
// ...
|
|
||||||
}
|
|
||||||
|
|
||||||
// domain/file.go
|
// domain/file.go
|
||||||
type FileListParams struct {
|
type FileListParams struct {
|
||||||
Filter string // raw DSL string
|
Filter string // raw DSL string
|
||||||
@@ -279,42 +295,38 @@ type FileListParams struct {
|
|||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
|
The DSL grammar itself is documented in `openapi.yaml` (the `filter` query
|
||||||
|
parameter), so the contract stays in one place.
|
||||||
|
|
||||||
### JWT Structure
|
### JWT Structure
|
||||||
|
|
||||||
```go
|
```go
|
||||||
type Claims struct {
|
type Claims struct {
|
||||||
jwt.RegisteredClaims
|
jwt.RegisteredClaims
|
||||||
UserID int16 `json:"uid"`
|
UserID int16 `json:"uid"`
|
||||||
IsAdmin bool `json:"adm"`
|
IsAdmin bool `json:"adm"`
|
||||||
SessionID int `json:"sid"`
|
SessionID int `json:"sid"`
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
Access token: short-lived (15 min). Refresh token: long-lived (30 days),
|
Access token: short-lived (15 min default). Refresh token: long-lived (30 days
|
||||||
stored as hash in `activity.sessions.token_hash`.
|
default), rotated on use, stored as a hash in `activity.sessions`. A separate
|
||||||
|
**content token** (default 6 h) is a single-file capability minted for media
|
||||||
|
URLs, so a long video keeps streaming past access-token expiry — see
|
||||||
|
`CONTENT_TOKEN_TTL` in `.env.example`.
|
||||||
|
|
||||||
|
### Perceptual Duplicate Detection
|
||||||
|
|
||||||
|
Images are dHash-ed inline on upload (`internal/imagehash`); video hashes are
|
||||||
|
backfilled by the `dedup` CLI (ffmpeg stays off the upload path). A rescan
|
||||||
|
rebuilds `data.duplicate_pairs` by inserting every pair within
|
||||||
|
`DUPLICATE_HASH_THRESHOLD` Hamming distance (BK-tree lookups, not O(N²)); the
|
||||||
|
duplicates API then groups pairs into connected-component clusters. See
|
||||||
|
`service/duplicate_service.go` and `service/duplicate_index.go`.
|
||||||
|
|
||||||
### Configuration (.env)
|
### Configuration (.env)
|
||||||
|
|
||||||
```env
|
Every variable the server reads is documented in `.env.example` (1:1 with
|
||||||
# Server
|
`config.Config`). Required at startup: `JWT_SECRET`, `ADMIN_PASSWORD`,
|
||||||
LISTEN_ADDR=:8080
|
`DATABASE_URL`, `FILES_PATH`, `THUMBS_CACHE_PATH`, `IMPORT_PATH`. Everything
|
||||||
JWT_SECRET=<random-32-bytes>
|
else has a sensible default (see `config.go`).
|
||||||
JWT_ACCESS_TTL=15m
|
|
||||||
JWT_REFRESH_TTL=720h
|
|
||||||
|
|
||||||
# Database
|
|
||||||
DATABASE_URL=postgres://user:pass@host:5432/tanabata?sslmode=disable
|
|
||||||
|
|
||||||
# Storage
|
|
||||||
FILES_PATH=/data/files
|
|
||||||
THUMBS_CACHE_PATH=/data/thumbs
|
|
||||||
|
|
||||||
# Thumbnails
|
|
||||||
THUMB_WIDTH=160
|
|
||||||
THUMB_HEIGHT=160
|
|
||||||
PREVIEW_WIDTH=1920
|
|
||||||
PREVIEW_HEIGHT=1080
|
|
||||||
|
|
||||||
# Import
|
|
||||||
IMPORT_PATH=/data/import
|
|
||||||
```
|
|
||||||
|
|||||||
@@ -0,0 +1,179 @@
|
|||||||
|
# Tanabata File Manager — Requirements
|
||||||
|
|
||||||
|
> Product requirements for Tanabata File Manager (TFM). Architecture and code
|
||||||
|
> layout are described separately in [ARCHITECTURE.md](ARCHITECTURE.md),
|
||||||
|
> [GO_PROJECT_STRUCTURE.md](GO_PROJECT_STRUCTURE.md) and
|
||||||
|
> [FRONTEND_STRUCTURE.md](FRONTEND_STRUCTURE.md).
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
Tanabata File Manager (TFM) is a multi-user, tag-based web file manager. It runs
|
||||||
|
on a client–server architecture and is operated entirely through a web
|
||||||
|
interface. Its goal is centralized, server-side storage of files with access and
|
||||||
|
management from both desktop and mobile browsers. The application is primarily
|
||||||
|
oriented toward **images and video**.
|
||||||
|
|
||||||
|
The app is a PWA that can be installed on a desktop or a phone. Files managed by
|
||||||
|
Tanabata are stored flat in a single directory; each file's on-disk name equals
|
||||||
|
its UUID in the database.
|
||||||
|
|
||||||
|
Support for additional database engines is planned for future versions.
|
||||||
|
|
||||||
|
## Core Concepts
|
||||||
|
|
||||||
|
- **File** — a single file on the server. It may carry any number of tags and
|
||||||
|
belong to any number of pools. It has a creator and optional access settings
|
||||||
|
(a user — which may be null, making the file public — plus read and edit
|
||||||
|
permission flags). It has an original name and metadata (key–value, including
|
||||||
|
all EXIF data).
|
||||||
|
- **Tag** — a label on a file. It may be attached to any number of files and
|
||||||
|
belong to at most one category. It has a name, a description, key–value
|
||||||
|
metadata, and may define auto-tag rules.
|
||||||
|
- **Auto-tag (tag rule)** — a rule stating that when tag A is attached to a file,
|
||||||
|
tag B is attached to the same file automatically.
|
||||||
|
- **Category** — an entity that logically groups several tags. It has a name, a
|
||||||
|
description, and key–value metadata.
|
||||||
|
- **Pool** — a logical grouping of files. It has a name, a description, and
|
||||||
|
key–value metadata. Files in a pool can be sorted automatically or arranged in
|
||||||
|
a user-defined manual order.
|
||||||
|
|
||||||
|
## Functional Requirements
|
||||||
|
|
||||||
|
### 1. File management
|
||||||
|
|
||||||
|
1. Browse the file list (lazy load, pagination).
|
||||||
|
2. Filter files by tags and metadata.
|
||||||
|
3. View and edit sort settings (persisted per user).
|
||||||
|
4. Multi-select files (Ctrl, Shift) and act on the selection:
|
||||||
|
1. Attach / detach tags.
|
||||||
|
2. Copy / paste tags.
|
||||||
|
3. Add to a pool.
|
||||||
|
4. View and edit access settings.
|
||||||
|
5. Delete (with a confirmation prompt).
|
||||||
|
5. View a single file.
|
||||||
|
6. Single-file actions:
|
||||||
|
1. Attach / detach tags.
|
||||||
|
2. Copy / paste tags.
|
||||||
|
3. Add to a pool.
|
||||||
|
4. View and edit access settings.
|
||||||
|
5. Replace the file (upload new content under the same ID).
|
||||||
|
6. Delete (with a confirmation prompt).
|
||||||
|
7. Browse files gallery-style (prev/next paging through the viewer).
|
||||||
|
8. Upload new files through the web UI (form or drag-and-drop onto the list).
|
||||||
|
9. Import new files from a folder on the server.
|
||||||
|
10. Near-duplicate detection for images and video:
|
||||||
|
1. Show groups (clusters) of duplicates.
|
||||||
|
2. Dismiss false duplicates (the app remembers that file A is _not_ a
|
||||||
|
duplicate of file B).
|
||||||
|
3. Choose which duplicate to keep and which to delete.
|
||||||
|
4. Choose, per field, which duplicate the surviving file inherits it from.
|
||||||
|
11. Trash:
|
||||||
|
1. Browse trashed files.
|
||||||
|
2. Restore from trash.
|
||||||
|
3. Delete permanently.
|
||||||
|
|
||||||
|
### 2. Tag management
|
||||||
|
|
||||||
|
1. Browse the tag list (lazy load, pagination).
|
||||||
|
2. Search by name.
|
||||||
|
3. View and edit sort settings (persisted per user).
|
||||||
|
4. Multi-select tags (Ctrl, Shift) and act on the selection:
|
||||||
|
1. Assign auto-tag rules.
|
||||||
|
2. Change category.
|
||||||
|
3. Delete (with a confirmation prompt).
|
||||||
|
5. View a single tag.
|
||||||
|
6. Single-tag actions:
|
||||||
|
1. Edit name, description, and metadata (key–value).
|
||||||
|
2. Change category.
|
||||||
|
3. Assign auto-tag rules.
|
||||||
|
4. Delete (with a confirmation prompt).
|
||||||
|
7. Create a tag:
|
||||||
|
1. Enter name, description, and metadata (key–value).
|
||||||
|
2. Assign a category (optional).
|
||||||
|
3. Assign auto-tag rules.
|
||||||
|
|
||||||
|
### 3. Category management
|
||||||
|
|
||||||
|
1. Browse the category list (lazy load, pagination).
|
||||||
|
2. Search by name.
|
||||||
|
3. View and edit sort settings (persisted per user).
|
||||||
|
4. Multi-select categories (Ctrl, Shift) and act on the selection:
|
||||||
|
1. View shared tags and tags attached to some (but not all) of them.
|
||||||
|
2. Attach / detach tags.
|
||||||
|
3. Delete (with a confirmation prompt).
|
||||||
|
5. View a single category.
|
||||||
|
6. Single-category actions:
|
||||||
|
1. Edit name, description, and metadata (key–value).
|
||||||
|
2. View attached tags.
|
||||||
|
3. Attach / detach tags.
|
||||||
|
4. Delete (with a confirmation prompt).
|
||||||
|
7. Create a category:
|
||||||
|
1. Enter name, description, and metadata (key–value).
|
||||||
|
2. Attach tags.
|
||||||
|
|
||||||
|
### 4. Pool management
|
||||||
|
|
||||||
|
1. Browse the pool list (lazy load, pagination).
|
||||||
|
2. Search by name.
|
||||||
|
3. View and edit sort settings (persisted per user).
|
||||||
|
4. Multi-select pools (Ctrl, Shift) and act on the selection:
|
||||||
|
1. View and edit access settings.
|
||||||
|
2. Delete (with a confirmation prompt).
|
||||||
|
5. View a single pool.
|
||||||
|
6. Single-pool actions:
|
||||||
|
1. Edit name, description, and metadata (key–value).
|
||||||
|
2. View and edit access settings.
|
||||||
|
3. View all files in the pool.
|
||||||
|
4. Filter the pool's files by tags.
|
||||||
|
5. Change the file sort setting (including disabling automatic sorting).
|
||||||
|
6. Reorder files manually (when automatic sorting is disabled).
|
||||||
|
7. Delete (with a confirmation prompt).
|
||||||
|
7. Create a pool:
|
||||||
|
1. Enter name, description, and metadata (key–value).
|
||||||
|
2. Attach files.
|
||||||
|
|
||||||
|
### 5. User settings
|
||||||
|
|
||||||
|
1. Username.
|
||||||
|
2. Password.
|
||||||
|
3. Sessions:
|
||||||
|
1. Terminate a session.
|
||||||
|
4. Path to the server folder scanned during file import.
|
||||||
|
|
||||||
|
### 6. Server administration (admin panel)
|
||||||
|
|
||||||
|
1. Users:
|
||||||
|
1. Browse the list.
|
||||||
|
2. View a single user.
|
||||||
|
3. Create.
|
||||||
|
4. Delete.
|
||||||
|
5. Block / unblock.
|
||||||
|
6. Set role (reader / editor).
|
||||||
|
|
||||||
|
### 7. Audit logging (in the database)
|
||||||
|
|
||||||
|
Log the following user actions:
|
||||||
|
|
||||||
|
1. File views.
|
||||||
|
2. Changes to file access settings.
|
||||||
|
3. Create / edit / delete of a file, tag, category, pool, or file–tag relation.
|
||||||
|
4. Create / block / unblock / delete of a user.
|
||||||
|
5. User role changes.
|
||||||
|
6. User login / logout.
|
||||||
|
7. Session termination.
|
||||||
|
|
||||||
|
## Non-Functional Requirements
|
||||||
|
|
||||||
|
1. The interface must be as simple and convenient as possible: everything needed
|
||||||
|
should be at hand, reachable in the fewest possible actions.
|
||||||
|
2. The interface must adapt to both desktop and mobile devices.
|
||||||
|
3. The interface must offer dark and light themes.
|
||||||
|
4. Use PWA technology, including a button that fully resets the PWA (except the
|
||||||
|
cache) and reloads it from the server.
|
||||||
|
5. Allow selected files to be cached and viewed offline in the installed PWA.
|
||||||
|
6. First-run setup must require minimal effort: automatic database migration, a
|
||||||
|
ready-made Docker Compose file, and a `.env` file with the configurable
|
||||||
|
installation parameters.
|
||||||
|
7. Use a Domain-Driven Design approach on the API server.
|
||||||
|
8. Reject files whose MIME type is not present in the database (no DB entry — no
|
||||||
|
support).
|
||||||
@@ -1,374 +0,0 @@
|
|||||||
from configparser import ConfigParser
|
|
||||||
from psycopg2.pool import ThreadedConnectionPool
|
|
||||||
from psycopg2.extras import RealDictCursor
|
|
||||||
from contextlib import contextmanager
|
|
||||||
from os import access, W_OK, makedirs, chmod, system
|
|
||||||
from os.path import isfile, join, basename
|
|
||||||
from shutil import move
|
|
||||||
from magic import Magic
|
|
||||||
from preview_generator.manager import PreviewManager
|
|
||||||
|
|
||||||
conf = None
|
|
||||||
|
|
||||||
mage = None
|
|
||||||
previewer = None
|
|
||||||
|
|
||||||
db_pool = None
|
|
||||||
|
|
||||||
DEFAULT_SORTING = {
|
|
||||||
"files": {
|
|
||||||
"key": "created",
|
|
||||||
"asc": False
|
|
||||||
},
|
|
||||||
"tags": {
|
|
||||||
"key": "created",
|
|
||||||
"asc": False
|
|
||||||
},
|
|
||||||
"categories": {
|
|
||||||
"key": "created",
|
|
||||||
"asc": False
|
|
||||||
},
|
|
||||||
"pools": {
|
|
||||||
"key": "created",
|
|
||||||
"asc": False
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def Initialize(conf_path="/etc/tfm/tfm.conf"):
|
|
||||||
global mage, previewer
|
|
||||||
load_config(conf_path)
|
|
||||||
mage = Magic(mime=True)
|
|
||||||
previewer = PreviewManager(conf["Paths"]["Thumbs"])
|
|
||||||
db_connect(conf["DB.limits"]["MinimumConnections"], conf["DB.limits"]["MaximumConnections"], **conf["DB.params"])
|
|
||||||
|
|
||||||
|
|
||||||
def load_config(path):
|
|
||||||
global conf
|
|
||||||
conf = ConfigParser()
|
|
||||||
conf.read(path)
|
|
||||||
|
|
||||||
|
|
||||||
def db_connect(minconn, maxconn, **kwargs):
|
|
||||||
global db_pool
|
|
||||||
db_pool = ThreadedConnectionPool(minconn, maxconn, **kwargs)
|
|
||||||
|
|
||||||
|
|
||||||
@contextmanager
|
|
||||||
def _db_cursor():
|
|
||||||
global db_pool
|
|
||||||
try:
|
|
||||||
conn = db_pool.getconn()
|
|
||||||
except:
|
|
||||||
raise RuntimeError("Database not connected")
|
|
||||||
try:
|
|
||||||
with conn.cursor(cursor_factory=RealDictCursor) as cur:
|
|
||||||
yield cur
|
|
||||||
conn.commit()
|
|
||||||
except:
|
|
||||||
conn.rollback()
|
|
||||||
raise
|
|
||||||
finally:
|
|
||||||
db_pool.putconn(conn)
|
|
||||||
|
|
||||||
|
|
||||||
def _validate_column_name(cur, table, column):
|
|
||||||
cur.execute("SELECT get_column_names(%s) AS name", (table,))
|
|
||||||
if all([column!=col["name"] for col in cur.fetchall()]):
|
|
||||||
raise RuntimeError("Invalid column name")
|
|
||||||
|
|
||||||
|
|
||||||
def authorize(username, password, useragent):
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
cur.execute("SELECT tfm_session_request(tfm_user_auth(%s, %s), %s) AS sid", (username, password, useragent))
|
|
||||||
sid = cur.fetchone()["sid"]
|
|
||||||
return TSession(sid)
|
|
||||||
|
|
||||||
|
|
||||||
class TSession:
|
|
||||||
sid = None
|
|
||||||
|
|
||||||
def __init__(self, sid):
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
cur.execute("SELECT tfm_session_validate(%s) IS NOT NULL AS valid", (sid,))
|
|
||||||
if not cur.fetchone()["valid"]:
|
|
||||||
raise RuntimeError("Invalid sid")
|
|
||||||
self.sid = sid
|
|
||||||
|
|
||||||
def terminate(self):
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
cur.execute("CALL tfm_session_terminate(%s)", (self.sid,))
|
|
||||||
del self
|
|
||||||
|
|
||||||
@property
|
|
||||||
def username(self):
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
cur.execute("SELECT tfm_session_username(%s) AS name", (self.sid,))
|
|
||||||
return cur.fetchone()["name"]
|
|
||||||
|
|
||||||
@property
|
|
||||||
def is_admin(self):
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
cur.execute("SELECT * FROM tfm_user_get_info(%s)", (self.sid,))
|
|
||||||
return cur.fetchone()["can_edit"]
|
|
||||||
|
|
||||||
def get_files(self, order_key=DEFAULT_SORTING["files"]["key"], order_asc=DEFAULT_SORTING["files"]["asc"], offset=0, limit=None):
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
_validate_column_name(cur, "v_files", order_key)
|
|
||||||
cur.execute("SELECT * FROM tfm_get_files(%%s) ORDER BY %s %s OFFSET %s LIMIT %s" % (
|
|
||||||
order_key,
|
|
||||||
"ASC" if order_asc else "DESC",
|
|
||||||
int(offset),
|
|
||||||
int(limit) if limit is not None else "ALL"
|
|
||||||
), (self.sid,))
|
|
||||||
return list(map(dict, cur.fetchall()))
|
|
||||||
|
|
||||||
def get_files_by_filter(self, philter=None, order_key=DEFAULT_SORTING["files"]["key"], order_asc=DEFAULT_SORTING["files"]["asc"], offset=0, limit=None):
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
_validate_column_name(cur, "v_files", order_key)
|
|
||||||
cur.execute("SELECT * FROM tfm_get_files_by_filter(%%s, %%s) ORDER BY %s %s OFFSET %s LIMIT %s" % (
|
|
||||||
order_key,
|
|
||||||
"ASC" if order_asc else "DESC",
|
|
||||||
int(offset),
|
|
||||||
int(limit) if limit is not None else "ALL"
|
|
||||||
), (self.sid, philter))
|
|
||||||
return list(map(dict, cur.fetchall()))
|
|
||||||
|
|
||||||
def get_tags(self, order_key=DEFAULT_SORTING["tags"]["key"], order_asc=DEFAULT_SORTING["tags"]["asc"], offset=0, limit=None):
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
_validate_column_name(cur, "v_tags", order_key)
|
|
||||||
cur.execute("SELECT * FROM tfm_get_tags(%%s) ORDER BY %s %s, name ASC OFFSET %s LIMIT %s" % (
|
|
||||||
order_key,
|
|
||||||
"ASC" if order_asc else "DESC",
|
|
||||||
int(offset),
|
|
||||||
int(limit) if limit is not None else "ALL"
|
|
||||||
), (self.sid,))
|
|
||||||
return list(map(dict, cur.fetchall()))
|
|
||||||
|
|
||||||
def get_categories(self, order_key=DEFAULT_SORTING["categories"]["key"], order_asc=DEFAULT_SORTING["categories"]["asc"], offset=0, limit=None):
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
_validate_column_name(cur, "v_categories", order_key)
|
|
||||||
cur.execute("SELECT * FROM tfm_get_categories(%%s) ORDER BY %s %s OFFSET %s LIMIT %s" % (
|
|
||||||
order_key,
|
|
||||||
"ASC" if order_asc else "DESC",
|
|
||||||
int(offset),
|
|
||||||
int(limit) if limit is not None else "ALL"
|
|
||||||
), (self.sid,))
|
|
||||||
return list(map(dict, cur.fetchall()))
|
|
||||||
|
|
||||||
def get_pools(self, order_key=DEFAULT_SORTING["pools"]["key"], order_asc=DEFAULT_SORTING["pools"]["asc"], offset=0, limit=None):
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
_validate_column_name(cur, "v_pools", order_key)
|
|
||||||
cur.execute("SELECT * FROM tfm_get_pools(%%s) ORDER BY %s %s OFFSET %s LIMIT %s" % (
|
|
||||||
order_key,
|
|
||||||
"ASC" if order_asc else "DESC",
|
|
||||||
int(offset),
|
|
||||||
int(limit) if limit is not None else "ALL"
|
|
||||||
), (self.sid,))
|
|
||||||
return list(map(dict, cur.fetchall()))
|
|
||||||
|
|
||||||
def get_autotags(self, order_key="child_id", order_asc=True, offset=0, limit=None):
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
_validate_column_name(cur, "v_autotags", order_key)
|
|
||||||
cur.execute("SELECT * FROM tfm_get_autotags(%%s) ORDER BY %s %s OFFSET %s LIMIT %s" % (
|
|
||||||
order_key,
|
|
||||||
"ASC" if order_asc else "DESC",
|
|
||||||
int(offset),
|
|
||||||
int(limit) if limit is not None else "ALL"
|
|
||||||
), (self.sid,))
|
|
||||||
return list(map(dict, cur.fetchall()))
|
|
||||||
|
|
||||||
def get_my_sessions(self, order_key="started", order_asc=False, offset=0, limit=None):
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
_validate_column_name(cur, "v_sessions", order_key)
|
|
||||||
cur.execute("SELECT * FROM tfm_get_my_sessions(%%s) ORDER BY %s %s OFFSET %s LIMIT %s" % (
|
|
||||||
order_key,
|
|
||||||
"ASC" if order_asc else "DESC",
|
|
||||||
int(offset),
|
|
||||||
int(limit) if limit is not None else "ALL"
|
|
||||||
), (self.sid,))
|
|
||||||
return list(map(dict, cur.fetchall()))
|
|
||||||
|
|
||||||
def get_tags_by_file(self, file_id, order_key=DEFAULT_SORTING["tags"]["key"], order_asc=DEFAULT_SORTING["tags"]["asc"], offset=0, limit=None):
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
_validate_column_name(cur, "v_tags", order_key)
|
|
||||||
cur.execute("SELECT * FROM tfm_get_tags_by_file(%%s, %%s) ORDER BY %s %s OFFSET %s LIMIT %s" % (
|
|
||||||
order_key,
|
|
||||||
"ASC" if order_asc else "DESC",
|
|
||||||
int(offset),
|
|
||||||
int(limit) if limit is not None else "ALL"
|
|
||||||
), (self.sid, file_id))
|
|
||||||
return list(map(dict, cur.fetchall()))
|
|
||||||
|
|
||||||
def get_files_by_tag(self, tag_id, order_key=DEFAULT_SORTING["files"]["key"], order_asc=DEFAULT_SORTING["files"]["asc"], offset=0, limit=None):
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
_validate_column_name(cur, "v_files", order_key)
|
|
||||||
cur.execute("SELECT * FROM tfm_get_files_by_tag(%%s, %%s) ORDER BY %s %s OFFSET %s LIMIT %s" % (
|
|
||||||
order_key,
|
|
||||||
"ASC" if order_asc else "DESC",
|
|
||||||
int(offset),
|
|
||||||
int(limit) if limit is not None else "ALL"
|
|
||||||
), (self.sid, tag_id))
|
|
||||||
return list(map(dict, cur.fetchall()))
|
|
||||||
|
|
||||||
def get_files_by_pool(self, pool_id, order_key=DEFAULT_SORTING["files"]["key"], order_asc=DEFAULT_SORTING["files"]["asc"], offset=0, limit=None):
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
_validate_column_name(cur, "v_files", order_key)
|
|
||||||
cur.execute("SELECT * FROM tfm_get_files_by_pool(%%s, %%s) ORDER BY %s %s OFFSET %s LIMIT %s" % (
|
|
||||||
order_key,
|
|
||||||
"ASC" if order_asc else "DESC",
|
|
||||||
int(offset),
|
|
||||||
int(limit) if limit is not None else "ALL"
|
|
||||||
), (self.sid, pool_id))
|
|
||||||
return list(map(dict, cur.fetchall()))
|
|
||||||
|
|
||||||
def get_parent_tags(self, tag_id, order_key=DEFAULT_SORTING["tags"]["key"], order_asc=DEFAULT_SORTING["tags"]["asc"], offset=0, limit=None):
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
_validate_column_name(cur, "v_tags", order_key)
|
|
||||||
cur.execute("SELECT * FROM tfm_get_parent_tags(%%s, %%s) ORDER BY %s %s OFFSET %s LIMIT %s" % (
|
|
||||||
order_key,
|
|
||||||
"ASC" if order_asc else "DESC",
|
|
||||||
int(offset),
|
|
||||||
int(limit) if limit is not None else "ALL"
|
|
||||||
), (self.sid, tag_id))
|
|
||||||
return list(map(dict, cur.fetchall()))
|
|
||||||
|
|
||||||
def get_my_file_views(self, file_id=None, order_key="datetime", order_asc=False, offset=0, limit=None):
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
_validate_column_name(cur, "v_files", order_key)
|
|
||||||
cur.execute("SELECT * FROM tfm_get_my_file_views(%%s, %%s) ORDER BY %s %s OFFSET %s LIMIT %s" % (
|
|
||||||
order_key,
|
|
||||||
"ASC" if order_asc else "DESC",
|
|
||||||
int(offset),
|
|
||||||
int(limit) if limit is not None else "ALL"
|
|
||||||
), (self.sid, file_id))
|
|
||||||
return list(map(dict, cur.fetchall()))
|
|
||||||
|
|
||||||
def get_file(self, file_id):
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
cur.execute("SELECT * FROM tfm_get_files(%s) WHERE id=%s", (self.sid, file_id))
|
|
||||||
return cur.fetchone()
|
|
||||||
|
|
||||||
def get_tag(self, tag_id):
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
cur.execute("SELECT * FROM tfm_get_tags(%s) WHERE id=%s", (self.sid, tag_id))
|
|
||||||
return cur.fetchone()
|
|
||||||
|
|
||||||
def get_category(self, category_id):
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
cur.execute("SELECT * FROM tfm_get_categories(%s) WHERE id=%s", (self.sid, category_id))
|
|
||||||
return cur.fetchone()
|
|
||||||
|
|
||||||
def view_file(self, file_id):
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
cur.execute("CALL tfm_view_file(%s, %s)", (self.sid, file_id))
|
|
||||||
|
|
||||||
def add_file(self, path, datetime=None, notes=None, is_private=None, orig_name=True):
|
|
||||||
if not isfile(path):
|
|
||||||
raise FileNotFoundError("No such file '%s'" % path)
|
|
||||||
if not access(conf["Paths"]["Files"], W_OK) or not access(conf["Paths"]["Thumbs"], W_OK):
|
|
||||||
raise PermissionError("Invalid directories for files and thumbs")
|
|
||||||
mime = mage.from_file(path)
|
|
||||||
if orig_name == True:
|
|
||||||
orig_name = basename(path)
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
cur.execute("SELECT * FROM tfm_add_file(%s, %s, %s, %s, %s, %s)", (self.sid, mime, datetime, notes, is_private, orig_name))
|
|
||||||
res = cur.fetchone()
|
|
||||||
file_id = res["f_id"]
|
|
||||||
ext = res["ext"]
|
|
||||||
file_path = join(conf["Paths"]["Files"], file_id)
|
|
||||||
move(path, file_path)
|
|
||||||
thumb_path = previewer.get_jpeg_preview(file_path, height=160, width=160)
|
|
||||||
preview_path = previewer.get_jpeg_preview(file_path, height=1080, width=1920)
|
|
||||||
chmod(file_path, 0o664)
|
|
||||||
chmod(thumb_path, 0o664)
|
|
||||||
chmod(preview_path, 0o664)
|
|
||||||
return file_id, ext
|
|
||||||
|
|
||||||
def add_tag(self, name, notes=None, color=None, category_id=None, is_private=None):
|
|
||||||
if color is not None:
|
|
||||||
color = color.replace('#', '')
|
|
||||||
if not category_id:
|
|
||||||
category_id = None
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
cur.execute("SELECT tfm_add_tag(%s, %s, %s, %s, %s, %s) AS id", (self.sid, name, notes, color, category_id, is_private))
|
|
||||||
return cur.fetchone()["id"]
|
|
||||||
|
|
||||||
def add_category(self, name, notes=None, color=None, is_private=None):
|
|
||||||
if color is not None:
|
|
||||||
color = color.replace('#', '')
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
cur.execute("SELECT tfm_add_category(%s, %s, %s, %s, %s) AS id", (self.sid, name, notes, color, is_private))
|
|
||||||
return cur.fetchone()["id"]
|
|
||||||
|
|
||||||
def add_pool(self, name, notes=None, parent_id=None, is_private=None):
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
cur.execute("SELECT tfm_add_pool(%s, %s, %s, %s, %s) AS id", (self.sid, name, notes, parent_id, is_private))
|
|
||||||
return cur.fetchone()["id"]
|
|
||||||
|
|
||||||
def add_autotag(self, child_id, parent_id, is_active=None, apply_to_existing=None):
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
cur.execute("SELECT tfm_add_autotag(%s, %s, %s, %s, %s) AS added", (self.sid, child_id, parent_id, is_active, apply_to_existing))
|
|
||||||
return cur.fetchone()["added"]
|
|
||||||
|
|
||||||
def add_file_to_tag(self, file_id, tag_id):
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
cur.execute("SELECT tfm_add_file_to_tag(%s, %s, %s) AS id", (self.sid, file_id, tag_id))
|
|
||||||
return list(map(lambda t: t["id"], cur.fetchall()))
|
|
||||||
|
|
||||||
def add_file_to_pool(self, file_id, pool_id):
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
cur.execute("SELECT tfm_add_file_to_pool(%s, %s, %s) AS added", (self.sid, file_id, pool_id))
|
|
||||||
return cur.fetchone()["added"]
|
|
||||||
|
|
||||||
def edit_file(self, file_id, mime=None, datetime=None, notes=None, is_private=None):
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
cur.execute("CALL tfm_edit_file(%s, %s, %s, %s, %s, %s)", (self.sid, file_id, mime, datetime, notes, is_private))
|
|
||||||
|
|
||||||
def edit_tag(self, tag_id, name=None, notes=None, color=None, category_id=None, is_private=None):
|
|
||||||
if color is not None:
|
|
||||||
color = color.replace('#', '')
|
|
||||||
if not category_id:
|
|
||||||
category_id = None
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
cur.execute("CALL tfm_edit_tag(%s, %s, %s, %s, %s, %s, %s)", (self.sid, tag_id, name, notes, color, category_id, is_private))
|
|
||||||
|
|
||||||
def edit_category(self, category_id, name=None, notes=None, color=None, is_private=None):
|
|
||||||
if color is not None:
|
|
||||||
color = color.replace('#', '')
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
cur.execute("CALL tfm_edit_category(%s, %s, %s, %s, %s, %s)", (self.sid, category_id, name, notes, color, is_private))
|
|
||||||
|
|
||||||
def edit_pool(self, pool_id, name=None, notes=None, parent_id=None, is_private=None):
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
cur.execute("CALL tfm_edit_pool(%s, %s, %s, %s, %s, %s)", (self.sid, pool_id, name, notes, parent_id, is_private))
|
|
||||||
|
|
||||||
def remove_file(self, file_id):
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
cur.execute("CALL tfm_remove_file(%s, %s)", (self.sid, file_id))
|
|
||||||
if system("rm %s/%s*" % (conf["Paths"]["Files"], file_id)):
|
|
||||||
raise RuntimeError("Failed to remove file '%s'" % file_id)
|
|
||||||
|
|
||||||
def remove_tag(self, tag_id):
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
cur.execute("CALL tfm_remove_tag(%s, %s)", (self.sid, tag_id))
|
|
||||||
|
|
||||||
def remove_category(self, category_id):
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
cur.execute("CALL tfm_remove_category(%s, %s)", (self.sid, category_id))
|
|
||||||
|
|
||||||
def remove_pool(self, pool_id):
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
cur.execute("CALL tfm_remove_pool(%s, %s)", (self.sid, pool_id))
|
|
||||||
|
|
||||||
def remove_autotag(self, child_id, parent_id):
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
cur.execute("CALL tfm_remove_autotag(%s, %s, %s)", (self.sid, child_id, parent_id))
|
|
||||||
|
|
||||||
def remove_file_to_tag(self, file_id, tag_id):
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
cur.execute("CALL tfm_remove_file_to_tag(%s, %s, %s)", (self.sid, file_id, tag_id))
|
|
||||||
|
|
||||||
def remove_file_to_pool(self, file_id, pool_id):
|
|
||||||
with _db_cursor() as cur:
|
|
||||||
cur.execute("CALL tfm_remove_file_to_pool(%s, %s, %s)", (self.sid, file_id, pool_id))
|
|
||||||
@@ -1,22 +0,0 @@
|
|||||||
package main
|
|
||||||
|
|
||||||
import (
|
|
||||||
"tanabata/internal/storage/postgres"
|
|
||||||
)
|
|
||||||
|
|
||||||
func main() {
|
|
||||||
postgres.InitDB("postgres://hiko:taikibansei@192.168.0.25/Tanabata_new?application_name=Tanabata%20testing")
|
|
||||||
// test_json := json.RawMessage([]byte("{\"valery\": \"ponosoff\"}"))
|
|
||||||
// data, statusCode, err := db.FileGetSlice(1, "", "+2", -2, 0)
|
|
||||||
// data, statusCode, err := db.FileGet(1, "0197d056-cfb0-76b5-97e0-bd588826393c")
|
|
||||||
// data, statusCode, err := db.FileAdd(1, "ABOBA.png", "image/png", time.Now(), "slkdfjsldkflsdkfj;sldkf", test_json)
|
|
||||||
// statusCode, err := db.FileUpdate(2, "0197d159-bf3a-7617-a3a8-a4a9fc39eca6", map[string]interface{}{
|
|
||||||
// "name": "ponos.png",
|
|
||||||
// })
|
|
||||||
// statusCode, err := db.FileDelete(1, "0197d155-848f-7221-ba4a-4660f257c7d5")
|
|
||||||
// v, e, err := postgres.FileGetAccess(1, "0197d15a-57f9-712c-991e-c512290e774f")
|
|
||||||
// fmt.Printf("V: %s, E: %s\n", v, e)
|
|
||||||
// fmt.Printf("Status: %d\n", statusCode)
|
|
||||||
// fmt.Printf("Error: %s\n", err)
|
|
||||||
// fmt.Printf("%+v\n", data)
|
|
||||||
}
|
|
||||||
@@ -1,22 +0,0 @@
|
|||||||
package main
|
|
||||||
|
|
||||||
import (
|
|
||||||
"fmt"
|
|
||||||
|
|
||||||
"tanabata/db"
|
|
||||||
)
|
|
||||||
|
|
||||||
func main() {
|
|
||||||
db.InitDB("postgres://hiko:taikibansei@192.168.0.25/Tanabata_new?application_name=Tanabata%20testing")
|
|
||||||
// test_json := json.RawMessage([]byte("{\"valery\": \"ponosoff\"}"))
|
|
||||||
// data, statusCode, err := db.FileGetSlice(2, "", "+2", -2, 0)
|
|
||||||
// data, statusCode, err := db.FileGet(1, "0197d056-cfb0-76b5-97e0-bd588826393c")
|
|
||||||
// data, statusCode, err := db.FileAdd(1, "ABOBA.png", "image/png", time.Now(), "slkdfjsldkflsdkfj;sldkf", test_json)
|
|
||||||
// statusCode, err := db.FileUpdate(2, "0197d159-bf3a-7617-a3a8-a4a9fc39eca6", map[string]interface{}{
|
|
||||||
// "name": "ponos.png",
|
|
||||||
// })
|
|
||||||
statusCode, err := db.FileDelete(1, "0197d155-848f-7221-ba4a-4660f257c7d5")
|
|
||||||
fmt.Printf("Status: %d\n", statusCode)
|
|
||||||
fmt.Printf("Error: %s\n", err)
|
|
||||||
// fmt.Printf("%+v\n", data)
|
|
||||||
}
|
|
||||||
@@ -1,79 +0,0 @@
|
|||||||
package db
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"errors"
|
|
||||||
"fmt"
|
|
||||||
"net/http"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/jackc/pgx/v5"
|
|
||||||
"github.com/jackc/pgx/v5/pgconn"
|
|
||||||
"github.com/jackc/pgx/v5/pgxpool"
|
|
||||||
)
|
|
||||||
|
|
||||||
var connPool *pgxpool.Pool
|
|
||||||
|
|
||||||
func InitDB(connString string) error {
|
|
||||||
poolConfig, err := pgxpool.ParseConfig(connString)
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("error while parsing connection string: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
poolConfig.MaxConns = 100
|
|
||||||
poolConfig.MinConns = 0
|
|
||||||
poolConfig.MaxConnLifetime = time.Hour
|
|
||||||
poolConfig.HealthCheckPeriod = 30 * time.Second
|
|
||||||
|
|
||||||
connPool, err = pgxpool.NewWithConfig(context.Background(), poolConfig)
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("error while initializing DB connections pool: %w", err)
|
|
||||||
}
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func transaction(handler func(context.Context, pgx.Tx) (statusCode int, err error)) (statusCode int, err error) {
|
|
||||||
ctx := context.Background()
|
|
||||||
tx, err := connPool.Begin(ctx)
|
|
||||||
if err != nil {
|
|
||||||
statusCode = http.StatusInternalServerError
|
|
||||||
return
|
|
||||||
}
|
|
||||||
statusCode, err = handler(ctx, tx)
|
|
||||||
if err != nil {
|
|
||||||
tx.Rollback(ctx)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
err = tx.Commit(ctx)
|
|
||||||
if err != nil {
|
|
||||||
statusCode = http.StatusInternalServerError
|
|
||||||
}
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
// Handle database error
|
|
||||||
func handleDBError(errIn error) (statusCode int, err error) {
|
|
||||||
if errIn == nil {
|
|
||||||
statusCode = http.StatusOK
|
|
||||||
return
|
|
||||||
}
|
|
||||||
if errors.Is(errIn, pgx.ErrNoRows) {
|
|
||||||
err = fmt.Errorf("not found")
|
|
||||||
statusCode = http.StatusNotFound
|
|
||||||
return
|
|
||||||
}
|
|
||||||
var pgErr *pgconn.PgError
|
|
||||||
if errors.As(errIn, &pgErr) {
|
|
||||||
switch pgErr.Code {
|
|
||||||
case "22P02", "22007": // Invalid data format
|
|
||||||
err = fmt.Errorf("%s", pgErr.Message)
|
|
||||||
statusCode = http.StatusBadRequest
|
|
||||||
return
|
|
||||||
case "23505": // Unique constraint violation
|
|
||||||
err = fmt.Errorf("already exists")
|
|
||||||
statusCode = http.StatusConflict
|
|
||||||
return
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return http.StatusInternalServerError, errIn
|
|
||||||
}
|
|
||||||
@@ -1,53 +0,0 @@
|
|||||||
package db
|
|
||||||
|
|
||||||
import (
|
|
||||||
"fmt"
|
|
||||||
"net/http"
|
|
||||||
"strconv"
|
|
||||||
"strings"
|
|
||||||
)
|
|
||||||
|
|
||||||
// Convert "filter" URL param to SQL "WHERE" condition
|
|
||||||
func filterToSQL(filter string) (sql string, statusCode int, err error) {
|
|
||||||
// filterTokens := strings.Split(string(filter), ";")
|
|
||||||
sql = "(true)"
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
// Convert "sort" URL param to SQL "ORDER BY"
|
|
||||||
func sortToSQL(sort string) (sql string, statusCode int, err error) {
|
|
||||||
if sort == "" {
|
|
||||||
return
|
|
||||||
}
|
|
||||||
sortOptions := strings.Split(sort, ",")
|
|
||||||
sql = " ORDER BY "
|
|
||||||
for i, sortOption := range sortOptions {
|
|
||||||
sortOrder := sortOption[:1]
|
|
||||||
sortColumn := sortOption[1:]
|
|
||||||
// parse sorting order marker
|
|
||||||
switch sortOrder {
|
|
||||||
case "+":
|
|
||||||
sortOrder = "ASC"
|
|
||||||
case "-":
|
|
||||||
sortOrder = "DESC"
|
|
||||||
default:
|
|
||||||
err = fmt.Errorf("invalid sorting order mark: %q", sortOrder)
|
|
||||||
statusCode = http.StatusBadRequest
|
|
||||||
return
|
|
||||||
}
|
|
||||||
// validate sorting column
|
|
||||||
var n int
|
|
||||||
n, err = strconv.Atoi(sortColumn)
|
|
||||||
if err != nil || n < 0 {
|
|
||||||
err = fmt.Errorf("invalid sorting column: %q", sortColumn)
|
|
||||||
statusCode = http.StatusBadRequest
|
|
||||||
return
|
|
||||||
}
|
|
||||||
// add sorting option to query
|
|
||||||
if i > 0 {
|
|
||||||
sql += ","
|
|
||||||
}
|
|
||||||
sql += fmt.Sprintf("%s %s NULLS LAST", sortColumn, sortOrder)
|
|
||||||
}
|
|
||||||
return
|
|
||||||
}
|
|
||||||
@@ -1,19 +0,0 @@
|
|||||||
module tanabata
|
|
||||||
|
|
||||||
go 1.23.0
|
|
||||||
|
|
||||||
toolchain go1.23.10
|
|
||||||
|
|
||||||
require github.com/jackc/pgx/v5 v5.7.5
|
|
||||||
|
|
||||||
require (
|
|
||||||
github.com/jackc/pgpassfile v1.0.0 // indirect
|
|
||||||
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
|
|
||||||
github.com/jackc/pgx v3.6.2+incompatible // indirect
|
|
||||||
github.com/jackc/puddle/v2 v2.2.2 // indirect
|
|
||||||
github.com/pkg/errors v0.9.1 // indirect
|
|
||||||
github.com/stretchr/testify v1.9.0 // indirect
|
|
||||||
golang.org/x/crypto v0.37.0 // indirect
|
|
||||||
golang.org/x/sync v0.13.0 // indirect
|
|
||||||
golang.org/x/text v0.24.0 // indirect
|
|
||||||
)
|
|
||||||
@@ -1,32 +0,0 @@
|
|||||||
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
|
||||||
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
|
||||||
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
|
||||||
github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM=
|
|
||||||
github.com/jackc/pgpassfile v1.0.0/go.mod h1:CEx0iS5ambNFdcRtxPj5JhEz+xB6uRky5eyVu/W2HEg=
|
|
||||||
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 h1:iCEnooe7UlwOQYpKFhBabPMi4aNAfoODPEFNiAnClxo=
|
|
||||||
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761/go.mod h1:5TJZWKEWniPve33vlWYSoGYefn3gLQRzjfDlhSJ9ZKM=
|
|
||||||
github.com/jackc/pgx v3.6.2+incompatible h1:2zP5OD7kiyR3xzRYMhOcXVvkDZsImVXfj+yIyTQf3/o=
|
|
||||||
github.com/jackc/pgx v3.6.2+incompatible/go.mod h1:0ZGrqGqkRlliWnWB4zKnWtjbSWbGkVEFm4TeybAXq+I=
|
|
||||||
github.com/jackc/pgx/v5 v5.7.5 h1:JHGfMnQY+IEtGM63d+NGMjoRpysB2JBwDr5fsngwmJs=
|
|
||||||
github.com/jackc/pgx/v5 v5.7.5/go.mod h1:aruU7o91Tc2q2cFp5h4uP3f6ztExVpyVv88Xl/8Vl8M=
|
|
||||||
github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
|
|
||||||
github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
|
|
||||||
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
|
|
||||||
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
|
|
||||||
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
|
||||||
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
|
||||||
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
|
|
||||||
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
|
|
||||||
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
|
|
||||||
github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
|
|
||||||
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
|
|
||||||
golang.org/x/crypto v0.37.0 h1:kJNSjF/Xp7kU0iB2Z+9viTPMW4EqqsrywMXLJOOsXSE=
|
|
||||||
golang.org/x/crypto v0.37.0/go.mod h1:vg+k43peMZ0pUMhYmVAWysMK35e6ioLh3wB8ZCAfbVc=
|
|
||||||
golang.org/x/sync v0.13.0 h1:AauUjRAJ9OSnvULf/ARrrVywoJDy0YS2AwQ98I37610=
|
|
||||||
golang.org/x/sync v0.13.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA=
|
|
||||||
golang.org/x/text v0.24.0 h1:dd5Bzh4yt5KYA8f9CJHCP4FB4D51c2c6JvN37xJJkJ0=
|
|
||||||
golang.org/x/text v0.24.0/go.mod h1:L8rBsPeo2pSS+xqN0d5u2ikmjtmoJbDBT1b7nHvFCdU=
|
|
||||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
|
||||||
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
|
||||||
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
|
||||||
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
|
||||||
@@ -1,122 +0,0 @@
|
|||||||
package domain
|
|
||||||
|
|
||||||
import (
|
|
||||||
"encoding/json"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/jackc/pgx/v5/pgtype"
|
|
||||||
)
|
|
||||||
|
|
||||||
type User struct {
|
|
||||||
Name string `json:"name"`
|
|
||||||
IsAdmin bool `json:"isAdmin"`
|
|
||||||
CanCreate bool `json:"canCreate"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type MIME struct {
|
|
||||||
Name string `json:"name"`
|
|
||||||
Extension string `json:"extension"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type (
|
|
||||||
CategoryCore struct {
|
|
||||||
ID string `json:"id"`
|
|
||||||
Name string `json:"name"`
|
|
||||||
Color pgtype.Text `json:"color"`
|
|
||||||
}
|
|
||||||
CategoryItem struct {
|
|
||||||
CategoryCore
|
|
||||||
}
|
|
||||||
CategoryFull struct {
|
|
||||||
CategoryCore
|
|
||||||
CreatedAt time.Time `json:"createdAt"`
|
|
||||||
Creator User `json:"creator"`
|
|
||||||
Notes pgtype.Text `json:"notes"`
|
|
||||||
}
|
|
||||||
)
|
|
||||||
|
|
||||||
type (
|
|
||||||
FileCore struct {
|
|
||||||
ID string `json:"id"`
|
|
||||||
Name pgtype.Text `json:"name"`
|
|
||||||
MIME MIME `json:"mime"`
|
|
||||||
}
|
|
||||||
FileItem struct {
|
|
||||||
FileCore
|
|
||||||
CreatedAt time.Time `json:"createdAt"`
|
|
||||||
Creator User `json:"creator"`
|
|
||||||
}
|
|
||||||
FileFull struct {
|
|
||||||
FileCore
|
|
||||||
CreatedAt time.Time `json:"createdAt"`
|
|
||||||
Creator User `json:"creator"`
|
|
||||||
Notes pgtype.Text `json:"notes"`
|
|
||||||
Metadata json.RawMessage `json:"metadata"`
|
|
||||||
Tags []TagCore `json:"tags"`
|
|
||||||
Viewed int `json:"viewed"`
|
|
||||||
}
|
|
||||||
)
|
|
||||||
|
|
||||||
type (
|
|
||||||
TagCore struct {
|
|
||||||
ID string `json:"id"`
|
|
||||||
Name string `json:"name"`
|
|
||||||
Color pgtype.Text `json:"color"`
|
|
||||||
}
|
|
||||||
TagItem struct {
|
|
||||||
TagCore
|
|
||||||
Category CategoryCore `json:"category"`
|
|
||||||
}
|
|
||||||
TagFull struct {
|
|
||||||
TagCore
|
|
||||||
Category CategoryCore `json:"category"`
|
|
||||||
CreatedAt time.Time `json:"createdAt"`
|
|
||||||
Creator User `json:"creator"`
|
|
||||||
Notes pgtype.Text `json:"notes"`
|
|
||||||
UsedIncl int `json:"usedIncl"`
|
|
||||||
UsedExcl int `json:"usedExcl"`
|
|
||||||
}
|
|
||||||
)
|
|
||||||
|
|
||||||
type Autotag struct {
|
|
||||||
TriggerTag TagCore `json:"triggerTag"`
|
|
||||||
AddTag TagCore `json:"addTag"`
|
|
||||||
IsActive bool `json:"isActive"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type (
|
|
||||||
PoolCore struct {
|
|
||||||
ID string `json:"id"`
|
|
||||||
Name string `json:"name"`
|
|
||||||
}
|
|
||||||
PoolItem struct {
|
|
||||||
PoolCore
|
|
||||||
}
|
|
||||||
PoolFull struct {
|
|
||||||
PoolCore
|
|
||||||
CreatedAt time.Time `json:"createdAt"`
|
|
||||||
Creator User `json:"creator"`
|
|
||||||
Notes pgtype.Text `json:"notes"`
|
|
||||||
Viewed int `json:"viewed"`
|
|
||||||
}
|
|
||||||
)
|
|
||||||
|
|
||||||
type Session struct {
|
|
||||||
ID int `json:"id"`
|
|
||||||
UserAgent string `json:"userAgent"`
|
|
||||||
StartedAt time.Time `json:"startedAt"`
|
|
||||||
ExpiresAt time.Time `json:"expiresAt"`
|
|
||||||
LastActivity time.Time `json:"lastActivity"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type Pagination struct {
|
|
||||||
Total int `json:"total"`
|
|
||||||
Offset int `json:"offset"`
|
|
||||||
Limit int `json:"limit"`
|
|
||||||
Count int `json:"count"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type Slice[T any] struct {
|
|
||||||
Pagination Pagination `json:"pagination"`
|
|
||||||
Data []T `json:"data"`
|
|
||||||
}
|
|
||||||
@@ -1,16 +0,0 @@
|
|||||||
package postgres
|
|
||||||
|
|
||||||
import "context"
|
|
||||||
|
|
||||||
func UserLogin(ctx context.Context, name, password string) (user_id int, err error) {
|
|
||||||
row := connPool.QueryRow(ctx, "SELECT id FROM users WHERE name=$1 AND password=crypt($2, password)", name, password)
|
|
||||||
err = row.Scan(&user_id)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
func UserAuth(ctx context.Context, user_id int) (ok, isAdmin bool) {
|
|
||||||
row := connPool.QueryRow(ctx, "SELECT is_admin FROM users WHERE id=$1", user_id)
|
|
||||||
err := row.Scan(&isAdmin)
|
|
||||||
ok = (err == nil)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
@@ -1,268 +0,0 @@
|
|||||||
package postgres
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"encoding/json"
|
|
||||||
"fmt"
|
|
||||||
"net/http"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/jackc/pgx/v5"
|
|
||||||
"github.com/jackc/pgx/v5/pgxpool"
|
|
||||||
|
|
||||||
"tanabata/internal/domain"
|
|
||||||
)
|
|
||||||
|
|
||||||
type FileStore struct {
|
|
||||||
db *pgxpool.Pool
|
|
||||||
}
|
|
||||||
|
|
||||||
func NewFileStore(db *pgxpool.Pool) *FileStore {
|
|
||||||
return &FileStore{db: db}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Get user's access rights to file
|
|
||||||
func (s *FileStore) getAccess(user_id int, file_id string) (canView, canEdit bool, err error) {
|
|
||||||
ctx := context.Background()
|
|
||||||
row := connPool.QueryRow(ctx, `
|
|
||||||
SELECT
|
|
||||||
COALESCE(a.view, FALSE) OR f.creator_id=$1 OR COALESCE(u.is_admin, FALSE),
|
|
||||||
COALESCE(a.edit, FALSE) OR f.creator_id=$1 OR COALESCE(u.is_admin, FALSE)
|
|
||||||
FROM data.files f
|
|
||||||
LEFT JOIN acl.files a ON a.file_id=f.id AND a.user_id=$1
|
|
||||||
LEFT JOIN system.users u ON u.id=$1
|
|
||||||
WHERE f.id=$2
|
|
||||||
`, user_id, file_id)
|
|
||||||
err = row.Scan(&canView, &canEdit)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
// Get a set of files
|
|
||||||
func (s *FileStore) GetSlice(user_id int, filter, sort string, limit, offset int) (files domain.Slice[domain.FileItem], statusCode int, err error) {
|
|
||||||
filterCond, statusCode, err := filterToSQL(filter)
|
|
||||||
if err != nil {
|
|
||||||
return
|
|
||||||
}
|
|
||||||
sortExpr, statusCode, err := sortToSQL(sort)
|
|
||||||
if err != nil {
|
|
||||||
return
|
|
||||||
}
|
|
||||||
// prepare query
|
|
||||||
query := `
|
|
||||||
SELECT
|
|
||||||
f.id,
|
|
||||||
f.name,
|
|
||||||
m.name,
|
|
||||||
m.extension,
|
|
||||||
uuid_extract_timestamp(f.id),
|
|
||||||
u.name,
|
|
||||||
u.is_admin
|
|
||||||
FROM data.files f
|
|
||||||
JOIN system.mime m ON m.id=f.mime_id
|
|
||||||
JOIN system.users u ON u.id=f.creator_id
|
|
||||||
WHERE NOT f.is_deleted AND (f.creator_id=$1 OR (SELECT view FROM acl.files WHERE file_id=f.id AND user_id=$1) OR (SELECT is_admin FROM system.users WHERE id=$1)) AND
|
|
||||||
`
|
|
||||||
query += filterCond
|
|
||||||
queryCount := query
|
|
||||||
query += sortExpr
|
|
||||||
if limit >= 0 {
|
|
||||||
query += fmt.Sprintf(" LIMIT %d", limit)
|
|
||||||
}
|
|
||||||
if offset > 0 {
|
|
||||||
query += fmt.Sprintf(" OFFSET %d", offset)
|
|
||||||
}
|
|
||||||
// execute query
|
|
||||||
statusCode, err = transaction(func(ctx context.Context, tx pgx.Tx) (statusCode int, err error) {
|
|
||||||
rows, err := tx.Query(ctx, query, user_id)
|
|
||||||
if err != nil {
|
|
||||||
statusCode, err = handleDBError(err)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
defer rows.Close()
|
|
||||||
count := 0
|
|
||||||
for rows.Next() {
|
|
||||||
var file domain.FileItem
|
|
||||||
err = rows.Scan(&file.ID, &file.Name, &file.MIME.Name, &file.MIME.Extension, &file.CreatedAt, &file.Creator.Name, &file.Creator.IsAdmin)
|
|
||||||
if err != nil {
|
|
||||||
statusCode = http.StatusInternalServerError
|
|
||||||
return
|
|
||||||
}
|
|
||||||
files.Data = append(files.Data, file)
|
|
||||||
count++
|
|
||||||
}
|
|
||||||
err = rows.Err()
|
|
||||||
if err != nil {
|
|
||||||
statusCode = http.StatusInternalServerError
|
|
||||||
return
|
|
||||||
}
|
|
||||||
files.Pagination.Limit = limit
|
|
||||||
files.Pagination.Offset = offset
|
|
||||||
files.Pagination.Count = count
|
|
||||||
row := tx.QueryRow(ctx, fmt.Sprintf("SELECT COUNT(*) FROM (%s) tmp", queryCount), user_id)
|
|
||||||
err = row.Scan(&files.Pagination.Total)
|
|
||||||
if err != nil {
|
|
||||||
statusCode = http.StatusInternalServerError
|
|
||||||
}
|
|
||||||
return
|
|
||||||
})
|
|
||||||
if err == nil {
|
|
||||||
statusCode = http.StatusOK
|
|
||||||
}
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
// Get file
|
|
||||||
func (s *FileStore) Get(user_id int, file_id string) (file domain.FileFull, statusCode int, err error) {
|
|
||||||
ctx := context.Background()
|
|
||||||
row := connPool.QueryRow(ctx, `
|
|
||||||
SELECT
|
|
||||||
f.id,
|
|
||||||
f.name,
|
|
||||||
m.name,
|
|
||||||
m.extension,
|
|
||||||
uuid_extract_timestamp(f.id),
|
|
||||||
u.name,
|
|
||||||
u.is_admin,
|
|
||||||
f.notes,
|
|
||||||
f.metadata,
|
|
||||||
(SELECT COUNT(*) FROM activity.file_views fv WHERE fv.file_id=$2 AND fv.user_id=$1)
|
|
||||||
FROM data.files f
|
|
||||||
JOIN system.mime m ON m.id=f.mime_id
|
|
||||||
JOIN system.users u ON u.id=f.creator_id
|
|
||||||
WHERE NOT f.is_deleted AND f.id=$2 AND (f.creator_id=$1 OR (SELECT view FROM acl.files WHERE file_id=$2 AND user_id=$1) OR (SELECT is_admin FROM system.users WHERE id=$1))
|
|
||||||
`, user_id, file_id)
|
|
||||||
err = row.Scan(&file.ID, &file.Name, &file.MIME.Name, &file.MIME.Extension, &file.CreatedAt, &file.Creator.Name, &file.Creator.IsAdmin, &file.Notes, &file.Metadata, &file.Viewed)
|
|
||||||
if err != nil {
|
|
||||||
statusCode, err = handleDBError(err)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
rows, err := connPool.Query(ctx, `
|
|
||||||
SELECT
|
|
||||||
t.id,
|
|
||||||
t.name,
|
|
||||||
COALESCE(t.color, c.color)
|
|
||||||
FROM data.tags t
|
|
||||||
LEFT JOIN data.categories c ON c.id=t.category_id
|
|
||||||
JOIN data.file_tag ft ON ft.tag_id=t.id
|
|
||||||
WHERE ft.file_id=$1
|
|
||||||
`, file_id)
|
|
||||||
if err != nil {
|
|
||||||
statusCode, err = handleDBError(err)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
defer rows.Close()
|
|
||||||
for rows.Next() {
|
|
||||||
var tag domain.TagCore
|
|
||||||
err = rows.Scan(&tag.ID, &tag.Name, &tag.Color)
|
|
||||||
if err != nil {
|
|
||||||
statusCode = http.StatusInternalServerError
|
|
||||||
return
|
|
||||||
}
|
|
||||||
file.Tags = append(file.Tags, tag)
|
|
||||||
}
|
|
||||||
err = rows.Err()
|
|
||||||
if err != nil {
|
|
||||||
statusCode = http.StatusInternalServerError
|
|
||||||
return
|
|
||||||
}
|
|
||||||
statusCode = http.StatusOK
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
// Add file
|
|
||||||
func (s *FileStore) Add(user_id int, name, mime string, datetime time.Time, notes string, metadata json.RawMessage) (file domain.FileCore, statusCode int, err error) {
|
|
||||||
ctx := context.Background()
|
|
||||||
var mime_id int
|
|
||||||
var extension string
|
|
||||||
row := connPool.QueryRow(ctx, "SELECT id, extension FROM system.mime WHERE name=$1", mime)
|
|
||||||
err = row.Scan(&mime_id, &extension)
|
|
||||||
if err != nil {
|
|
||||||
if err == pgx.ErrNoRows {
|
|
||||||
err = fmt.Errorf("unsupported file type: %q", mime)
|
|
||||||
statusCode = http.StatusBadRequest
|
|
||||||
} else {
|
|
||||||
statusCode, err = handleDBError(err)
|
|
||||||
}
|
|
||||||
return
|
|
||||||
}
|
|
||||||
row = connPool.QueryRow(ctx, `
|
|
||||||
INSERT INTO data.files (name, mime_id, datetime, creator_id, notes, metadata)
|
|
||||||
VALUES (NULLIF($1, ''), $2, $3, $4, NULLIF($5 ,''), $6)
|
|
||||||
RETURNING id
|
|
||||||
`, name, mime_id, datetime, user_id, notes, metadata)
|
|
||||||
err = row.Scan(&file.ID)
|
|
||||||
if err != nil {
|
|
||||||
statusCode, err = handleDBError(err)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
file.Name.String = name
|
|
||||||
file.Name.Valid = (name != "")
|
|
||||||
file.MIME.Name = mime
|
|
||||||
file.MIME.Extension = extension
|
|
||||||
statusCode = http.StatusOK
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
// Update file
|
|
||||||
func (s *FileStore) Update(user_id int, file_id string, updates map[string]interface{}) (statusCode int, err error) {
|
|
||||||
if len(updates) == 0 {
|
|
||||||
err = fmt.Errorf("no fields provided for update")
|
|
||||||
statusCode = http.StatusBadRequest
|
|
||||||
return
|
|
||||||
}
|
|
||||||
writableFields := map[string]bool{
|
|
||||||
"name": true,
|
|
||||||
"datetime": true,
|
|
||||||
"notes": true,
|
|
||||||
"metadata": true,
|
|
||||||
}
|
|
||||||
query := "UPDATE data.files SET"
|
|
||||||
newValues := []interface{}{user_id}
|
|
||||||
count := 2
|
|
||||||
for field, value := range updates {
|
|
||||||
if !writableFields[field] {
|
|
||||||
err = fmt.Errorf("invalid field: %q", field)
|
|
||||||
statusCode = http.StatusBadRequest
|
|
||||||
return
|
|
||||||
}
|
|
||||||
query += fmt.Sprintf(" %s=NULLIF($%d, '')", field, count)
|
|
||||||
newValues = append(newValues, value)
|
|
||||||
count++
|
|
||||||
}
|
|
||||||
query += fmt.Sprintf(
|
|
||||||
" WHERE id=$%d AND (creator_id=$1 OR (SELECT edit FROM acl.files WHERE file_id=$%d AND user_id=$1) OR (SELECT is_admin FROM system.users WHERE id=$1))",
|
|
||||||
count, count)
|
|
||||||
newValues = append(newValues, file_id)
|
|
||||||
ctx := context.Background()
|
|
||||||
commandTag, err := connPool.Exec(ctx, query, newValues...)
|
|
||||||
if err != nil {
|
|
||||||
statusCode, err = handleDBError(err)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
if commandTag.RowsAffected() == 0 {
|
|
||||||
err = fmt.Errorf("not found")
|
|
||||||
statusCode = http.StatusNotFound
|
|
||||||
return
|
|
||||||
}
|
|
||||||
statusCode = http.StatusNoContent
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
// Delete file
|
|
||||||
func (s *FileStore) Delete(user_id int, file_id string) (statusCode int, err error) {
|
|
||||||
ctx := context.Background()
|
|
||||||
commandTag, err := connPool.Exec(ctx,
|
|
||||||
"DELETE FROM data.files WHERE id=$2 AND (creator_id=$1 OR (SELECT edit FROM acl.files WHERE file_id=$2 AND user_id=$1) OR (SELECT is_admin FROM system.users WHERE id=$1))",
|
|
||||||
user_id, file_id)
|
|
||||||
if err != nil {
|
|
||||||
statusCode, err = handleDBError(err)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
if commandTag.RowsAffected() == 0 {
|
|
||||||
err = fmt.Errorf("not found")
|
|
||||||
statusCode = http.StatusNotFound
|
|
||||||
return
|
|
||||||
}
|
|
||||||
statusCode = http.StatusNoContent
|
|
||||||
return
|
|
||||||
}
|
|
||||||
@@ -1,92 +0,0 @@
|
|||||||
package postgres
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"errors"
|
|
||||||
"fmt"
|
|
||||||
"net/http"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/jackc/pgx/v5"
|
|
||||||
"github.com/jackc/pgx/v5/pgconn"
|
|
||||||
"github.com/jackc/pgx/v5/pgxpool"
|
|
||||||
)
|
|
||||||
|
|
||||||
type Storage struct {
|
|
||||||
db *pgxpool.Pool
|
|
||||||
}
|
|
||||||
|
|
||||||
var connPool *pgxpool.Pool
|
|
||||||
|
|
||||||
// Initialize new database storage
|
|
||||||
func New(dbURL string) (*Storage, error) {
|
|
||||||
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
|
||||||
defer cancel()
|
|
||||||
config, err := pgxpool.ParseConfig(dbURL)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("failed to parse DB URL: %w", err)
|
|
||||||
}
|
|
||||||
config.MaxConns = 10
|
|
||||||
config.MinConns = 2
|
|
||||||
config.HealthCheckPeriod = time.Minute
|
|
||||||
db, err := pgxpool.NewWithConfig(ctx, config)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("failed to connect to database: %w", err)
|
|
||||||
}
|
|
||||||
err = db.Ping(ctx)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("database ping failed: %w", err)
|
|
||||||
}
|
|
||||||
return &Storage{db: db}, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// Close database storage
|
|
||||||
func (s *Storage) Close() {
|
|
||||||
s.db.Close()
|
|
||||||
}
|
|
||||||
|
|
||||||
// Run handler inside transaction
|
|
||||||
func (s *Storage) transaction(ctx context.Context, handler func(context.Context, pgx.Tx) (statusCode int, err error)) (statusCode int, err error) {
|
|
||||||
tx, err := connPool.Begin(ctx)
|
|
||||||
if err != nil {
|
|
||||||
statusCode = http.StatusInternalServerError
|
|
||||||
return
|
|
||||||
}
|
|
||||||
statusCode, err = handler(ctx, tx)
|
|
||||||
if err != nil {
|
|
||||||
tx.Rollback(ctx)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
err = tx.Commit(ctx)
|
|
||||||
if err != nil {
|
|
||||||
statusCode = http.StatusInternalServerError
|
|
||||||
}
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
// Handle database error
|
|
||||||
func (s *Storage) handleDBError(errIn error) (statusCode int, err error) {
|
|
||||||
if errIn == nil {
|
|
||||||
statusCode = http.StatusOK
|
|
||||||
return
|
|
||||||
}
|
|
||||||
if errors.Is(errIn, pgx.ErrNoRows) {
|
|
||||||
err = fmt.Errorf("not found")
|
|
||||||
statusCode = http.StatusNotFound
|
|
||||||
return
|
|
||||||
}
|
|
||||||
var pgErr *pgconn.PgError
|
|
||||||
if errors.As(errIn, &pgErr) {
|
|
||||||
switch pgErr.Code {
|
|
||||||
case "22P02", "22007": // Invalid data format
|
|
||||||
err = fmt.Errorf("%s", pgErr.Message)
|
|
||||||
statusCode = http.StatusBadRequest
|
|
||||||
return
|
|
||||||
case "23505": // Unique constraint violation
|
|
||||||
err = fmt.Errorf("already exists")
|
|
||||||
statusCode = http.StatusConflict
|
|
||||||
return
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return http.StatusInternalServerError, errIn
|
|
||||||
}
|
|
||||||
@@ -1,53 +0,0 @@
|
|||||||
package postgres
|
|
||||||
|
|
||||||
import (
|
|
||||||
"fmt"
|
|
||||||
"net/http"
|
|
||||||
"strconv"
|
|
||||||
"strings"
|
|
||||||
)
|
|
||||||
|
|
||||||
// Convert "filter" URL param to SQL "WHERE" condition
|
|
||||||
func filterToSQL(filter string) (sql string, statusCode int, err error) {
|
|
||||||
// filterTokens := strings.Split(string(filter), ";")
|
|
||||||
sql = "(true)"
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
// Convert "sort" URL param to SQL "ORDER BY"
|
|
||||||
func sortToSQL(sort string) (sql string, statusCode int, err error) {
|
|
||||||
if sort == "" {
|
|
||||||
return
|
|
||||||
}
|
|
||||||
sortOptions := strings.Split(sort, ",")
|
|
||||||
sql = " ORDER BY "
|
|
||||||
for i, sortOption := range sortOptions {
|
|
||||||
sortOrder := sortOption[:1]
|
|
||||||
sortColumn := sortOption[1:]
|
|
||||||
// parse sorting order marker
|
|
||||||
switch sortOrder {
|
|
||||||
case "+":
|
|
||||||
sortOrder = "ASC"
|
|
||||||
case "-":
|
|
||||||
sortOrder = "DESC"
|
|
||||||
default:
|
|
||||||
err = fmt.Errorf("invalid sorting order mark: %q", sortOrder)
|
|
||||||
statusCode = http.StatusBadRequest
|
|
||||||
return
|
|
||||||
}
|
|
||||||
// validate sorting column
|
|
||||||
var n int
|
|
||||||
n, err = strconv.Atoi(sortColumn)
|
|
||||||
if err != nil || n < 0 {
|
|
||||||
err = fmt.Errorf("invalid sorting column: %q", sortColumn)
|
|
||||||
statusCode = http.StatusBadRequest
|
|
||||||
return
|
|
||||||
}
|
|
||||||
// add sorting option to query
|
|
||||||
if i > 0 {
|
|
||||||
sql += ","
|
|
||||||
}
|
|
||||||
sql += fmt.Sprintf("%s %s NULLS LAST", sortColumn, sortOrder)
|
|
||||||
}
|
|
||||||
return
|
|
||||||
}
|
|
||||||
@@ -1,21 +0,0 @@
|
|||||||
package storage
|
|
||||||
|
|
||||||
import (
|
|
||||||
"encoding/json"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"tanabata/internal/domain"
|
|
||||||
)
|
|
||||||
|
|
||||||
type Storage interface {
|
|
||||||
FileRepository
|
|
||||||
Close()
|
|
||||||
}
|
|
||||||
|
|
||||||
type FileRepository interface {
|
|
||||||
GetSlice(user_id int, filter, sort string, limit, offset int) (files domain.Slice[domain.FileItem], statusCode int, err error)
|
|
||||||
Get(user_id int, file_id string) (file domain.FileFull, statusCode int, err error)
|
|
||||||
Add(user_id int, name, mime string, datetime time.Time, notes string, metadata json.RawMessage) (file domain.FileCore, statusCode int, err error)
|
|
||||||
Update(user_id int, file_id string, updates map[string]interface{}) (statusCode int, err error)
|
|
||||||
Delete(user_id int, file_id string) (statusCode int, err error)
|
|
||||||
}
|
|
||||||
@@ -1,120 +0,0 @@
|
|||||||
package models
|
|
||||||
|
|
||||||
import (
|
|
||||||
"encoding/json"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/jackc/pgx/v5/pgtype"
|
|
||||||
)
|
|
||||||
|
|
||||||
type User struct {
|
|
||||||
Name string `json:"name"`
|
|
||||||
IsAdmin bool `json:"isAdmin"`
|
|
||||||
CanCreate bool `json:"canCreate"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type MIME struct {
|
|
||||||
Name string `json:"name"`
|
|
||||||
Extension string `json:"extension"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type (
|
|
||||||
CategoryCore struct {
|
|
||||||
ID string `json:"id"`
|
|
||||||
Name string `json:"name"`
|
|
||||||
Color pgtype.Text `json:"color"`
|
|
||||||
}
|
|
||||||
CategoryItem struct {
|
|
||||||
CategoryCore
|
|
||||||
}
|
|
||||||
CategoryFull struct {
|
|
||||||
CategoryCore
|
|
||||||
CreatedAt time.Time `json:"createdAt"`
|
|
||||||
Creator User `json:"creator"`
|
|
||||||
Notes pgtype.Text `json:"notes"`
|
|
||||||
}
|
|
||||||
)
|
|
||||||
|
|
||||||
type (
|
|
||||||
FileCore struct {
|
|
||||||
ID string `json:"id"`
|
|
||||||
Name pgtype.Text `json:"name"`
|
|
||||||
MIME MIME `json:"mime"`
|
|
||||||
}
|
|
||||||
FileItem struct {
|
|
||||||
FileCore
|
|
||||||
CreatedAt time.Time `json:"createdAt"`
|
|
||||||
Creator User `json:"creator"`
|
|
||||||
}
|
|
||||||
FileFull struct {
|
|
||||||
FileCore
|
|
||||||
CreatedAt time.Time `json:"createdAt"`
|
|
||||||
Creator User `json:"creator"`
|
|
||||||
Notes pgtype.Text `json:"notes"`
|
|
||||||
Metadata json.RawMessage `json:"metadata"`
|
|
||||||
Tags []TagCore `json:"tags"`
|
|
||||||
Viewed int `json:"viewed"`
|
|
||||||
}
|
|
||||||
)
|
|
||||||
|
|
||||||
type (
|
|
||||||
TagCore struct {
|
|
||||||
ID string `json:"id"`
|
|
||||||
Name string `json:"name"`
|
|
||||||
Color pgtype.Text `json:"color"`
|
|
||||||
}
|
|
||||||
TagItem struct {
|
|
||||||
TagCore
|
|
||||||
Category CategoryCore `json:"category"`
|
|
||||||
}
|
|
||||||
TagFull struct {
|
|
||||||
TagCore
|
|
||||||
Category CategoryCore `json:"category"`
|
|
||||||
CreatedAt time.Time `json:"createdAt"`
|
|
||||||
Creator User `json:"creator"`
|
|
||||||
Notes pgtype.Text `json:"notes"`
|
|
||||||
}
|
|
||||||
)
|
|
||||||
|
|
||||||
type Autotag struct {
|
|
||||||
TriggerTag TagCore `json:"triggerTag"`
|
|
||||||
AddTag TagCore `json:"addTag"`
|
|
||||||
IsActive bool `json:"isActive"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type (
|
|
||||||
PoolCore struct {
|
|
||||||
ID string `json:"id"`
|
|
||||||
Name string `json:"name"`
|
|
||||||
}
|
|
||||||
PoolItem struct {
|
|
||||||
PoolCore
|
|
||||||
}
|
|
||||||
PoolFull struct {
|
|
||||||
PoolCore
|
|
||||||
CreatedAt time.Time `json:"createdAt"`
|
|
||||||
Creator User `json:"creator"`
|
|
||||||
Notes pgtype.Text `json:"notes"`
|
|
||||||
Viewed int `json:"viewed"`
|
|
||||||
}
|
|
||||||
)
|
|
||||||
|
|
||||||
type Session struct {
|
|
||||||
ID int `json:"id"`
|
|
||||||
UserAgent string `json:"userAgent"`
|
|
||||||
StartedAt time.Time `json:"startedAt"`
|
|
||||||
ExpiresAt time.Time `json:"expiresAt"`
|
|
||||||
LastActivity time.Time `json:"lastActivity"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type Pagination struct {
|
|
||||||
Total int `json:"total"`
|
|
||||||
Offset int `json:"offset"`
|
|
||||||
Limit int `json:"limit"`
|
|
||||||
Count int `json:"count"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type Slice[T any] struct {
|
|
||||||
Pagination Pagination `json:"pagination"`
|
|
||||||
Data []T `json:"data"`
|
|
||||||
}
|
|
||||||
@@ -1,36 +0,0 @@
|
|||||||
body {
|
|
||||||
justify-content: center;
|
|
||||||
align-items: center;
|
|
||||||
}
|
|
||||||
|
|
||||||
.decoration {
|
|
||||||
position: absolute;
|
|
||||||
top: 0;
|
|
||||||
}
|
|
||||||
|
|
||||||
.decoration.left {
|
|
||||||
left: 0;
|
|
||||||
width: 20vw;
|
|
||||||
}
|
|
||||||
|
|
||||||
.decoration.right {
|
|
||||||
right: 0;
|
|
||||||
width: 20vw;
|
|
||||||
}
|
|
||||||
|
|
||||||
#auth {
|
|
||||||
max-width: 100%;
|
|
||||||
}
|
|
||||||
|
|
||||||
#auth h1 {
|
|
||||||
margin-bottom: 28px;
|
|
||||||
}
|
|
||||||
|
|
||||||
#auth .form-control {
|
|
||||||
margin: 14px 0;
|
|
||||||
border-radius: 14px;
|
|
||||||
}
|
|
||||||
|
|
||||||
#login {
|
|
||||||
margin-top: 20px;
|
|
||||||
}
|
|
||||||
@@ -1,54 +0,0 @@
|
|||||||
html, body {
|
|
||||||
margin: 0;
|
|
||||||
padding: 0;
|
|
||||||
}
|
|
||||||
|
|
||||||
body {
|
|
||||||
background-color: #312F45;
|
|
||||||
color: #f0f0f0;
|
|
||||||
position: absolute;
|
|
||||||
top: 0;
|
|
||||||
left: 0;
|
|
||||||
bottom: 0;
|
|
||||||
right: 0;
|
|
||||||
min-height: 100vh;
|
|
||||||
display: flex;
|
|
||||||
flex-direction: column;
|
|
||||||
justify-content: space-between;
|
|
||||||
align-items: stretch;
|
|
||||||
font-family: Epilogue;
|
|
||||||
overflow-x: hidden;
|
|
||||||
}
|
|
||||||
|
|
||||||
.btn {
|
|
||||||
height: 50px;
|
|
||||||
width: 100%;
|
|
||||||
border-radius: 14px;
|
|
||||||
font-size: 1.5rem;
|
|
||||||
font-weight: 500;
|
|
||||||
}
|
|
||||||
|
|
||||||
.btn-primary {
|
|
||||||
background-color: #9592B5;
|
|
||||||
border-color: #454261;
|
|
||||||
}
|
|
||||||
|
|
||||||
.btn-primary:hover {
|
|
||||||
background-color: #7D7AA4;
|
|
||||||
border-color: #454261;
|
|
||||||
}
|
|
||||||
|
|
||||||
.btn-danger {
|
|
||||||
background-color: #DB6060;
|
|
||||||
border-color: #851E1E;
|
|
||||||
}
|
|
||||||
|
|
||||||
.btn-danger:hover {
|
|
||||||
background-color: #D64848;
|
|
||||||
border-color: #851E1E;
|
|
||||||
}
|
|
||||||
|
|
||||||
.btn-row {
|
|
||||||
display: flex;
|
|
||||||
justify-content: space-between;
|
|
||||||
}
|
|
||||||
@@ -1,388 +0,0 @@
|
|||||||
header, footer {
|
|
||||||
margin: 0;
|
|
||||||
padding: 10px;
|
|
||||||
box-sizing: border-box;
|
|
||||||
}
|
|
||||||
|
|
||||||
header {
|
|
||||||
padding: 20px;
|
|
||||||
box-shadow: 0 5px 5px #0004;
|
|
||||||
}
|
|
||||||
|
|
||||||
.icon-header {
|
|
||||||
height: .8em;
|
|
||||||
}
|
|
||||||
|
|
||||||
#select {
|
|
||||||
cursor: pointer;
|
|
||||||
}
|
|
||||||
|
|
||||||
.sorting {
|
|
||||||
position: relative;
|
|
||||||
display: flex;
|
|
||||||
justify-content: space-between;
|
|
||||||
}
|
|
||||||
|
|
||||||
#sorting {
|
|
||||||
cursor: pointer;
|
|
||||||
}
|
|
||||||
|
|
||||||
.highlighted {
|
|
||||||
color: #9999AD;
|
|
||||||
}
|
|
||||||
|
|
||||||
#icon-expand {
|
|
||||||
height: 10px;
|
|
||||||
}
|
|
||||||
|
|
||||||
#sorting-options {
|
|
||||||
position: absolute;
|
|
||||||
right: 0;
|
|
||||||
top: 114%;
|
|
||||||
padding: 4px 10px;
|
|
||||||
box-sizing: border-box;
|
|
||||||
background-color: #111118;
|
|
||||||
border-radius: 10px;
|
|
||||||
text-align: left;
|
|
||||||
box-shadow: 0 0 10px black;
|
|
||||||
z-index: 9999;
|
|
||||||
}
|
|
||||||
|
|
||||||
.sorting-option {
|
|
||||||
padding: 4px 0;
|
|
||||||
display: flex;
|
|
||||||
justify-content: space-between;
|
|
||||||
}
|
|
||||||
|
|
||||||
.sorting-option input[type="radio"] {
|
|
||||||
float: unset;
|
|
||||||
margin-left: 1.8em;
|
|
||||||
}
|
|
||||||
|
|
||||||
.filtering-wrapper {
|
|
||||||
margin-top: 10px;
|
|
||||||
}
|
|
||||||
|
|
||||||
.filtering-block {
|
|
||||||
position: absolute;
|
|
||||||
top: 128px;
|
|
||||||
left: 14px;
|
|
||||||
right: 14px;
|
|
||||||
padding: 14px;
|
|
||||||
background-color: #111118;
|
|
||||||
border-radius: 10px;
|
|
||||||
box-shadow: 0 0 10px 4px #0004;
|
|
||||||
z-index: 9998;
|
|
||||||
}
|
|
||||||
|
|
||||||
main {
|
|
||||||
margin: 0;
|
|
||||||
padding: 10px;
|
|
||||||
height: 100%;
|
|
||||||
display: flex;
|
|
||||||
justify-content: space-between;
|
|
||||||
align-content: flex-start;
|
|
||||||
align-items: flex-start;
|
|
||||||
flex-wrap: wrap;
|
|
||||||
box-sizing: border-box;
|
|
||||||
overflow-x: hidden;
|
|
||||||
overflow-y: scroll;
|
|
||||||
}
|
|
||||||
|
|
||||||
main:after {
|
|
||||||
content: "";
|
|
||||||
flex: auto;
|
|
||||||
}
|
|
||||||
|
|
||||||
.item-preview {
|
|
||||||
position: relative;
|
|
||||||
}
|
|
||||||
|
|
||||||
.item-selected:after {
|
|
||||||
content: "";
|
|
||||||
display: block;
|
|
||||||
position: absolute;
|
|
||||||
top: 10px;
|
|
||||||
right: 10px;
|
|
||||||
width: 100%;
|
|
||||||
height: 50%;
|
|
||||||
background-image: url("/static/images/icon-select.svg");
|
|
||||||
background-size: contain;
|
|
||||||
background-position: right;
|
|
||||||
background-repeat: no-repeat;
|
|
||||||
}
|
|
||||||
|
|
||||||
.file-preview {
|
|
||||||
margin: 1px 0;
|
|
||||||
padding: 0;
|
|
||||||
width: 160px;
|
|
||||||
height: 160px;
|
|
||||||
max-width: calc(33vw - 7px);
|
|
||||||
max-height: calc(33vw - 7px);
|
|
||||||
overflow: hidden;
|
|
||||||
cursor: pointer;
|
|
||||||
}
|
|
||||||
|
|
||||||
.file-thumb {
|
|
||||||
width: 100%;
|
|
||||||
height: 100%;
|
|
||||||
object-fit: contain;
|
|
||||||
object-position: center;
|
|
||||||
}
|
|
||||||
|
|
||||||
.file-preview .overlay {
|
|
||||||
position: absolute;
|
|
||||||
top: 0;
|
|
||||||
right: 0;
|
|
||||||
bottom: 0;
|
|
||||||
left: 0;
|
|
||||||
background-color: #0002;
|
|
||||||
}
|
|
||||||
|
|
||||||
.file-preview:hover .overlay {
|
|
||||||
background-color: #0004;
|
|
||||||
}
|
|
||||||
|
|
||||||
.tag-preview, .filtering-token {
|
|
||||||
margin: 5px 5px;
|
|
||||||
padding: 5px 10px;
|
|
||||||
border-radius: 5px;
|
|
||||||
background-color: #444455;
|
|
||||||
cursor: pointer;
|
|
||||||
}
|
|
||||||
|
|
||||||
.category-preview {
|
|
||||||
margin: 5px 5px;
|
|
||||||
padding: 5px 10px;
|
|
||||||
border-radius: 5px;
|
|
||||||
background-color: #444455;
|
|
||||||
cursor: pointer;
|
|
||||||
}
|
|
||||||
|
|
||||||
.file {
|
|
||||||
margin: 0;
|
|
||||||
padding: 0;
|
|
||||||
min-width: 100vw;
|
|
||||||
min-height: 100vh;
|
|
||||||
max-width: 100vw;
|
|
||||||
max-height: 100vh;
|
|
||||||
display: flex;
|
|
||||||
justify-content: center;
|
|
||||||
align-items: center;
|
|
||||||
background-color: black;
|
|
||||||
}
|
|
||||||
|
|
||||||
.file .preview-img {
|
|
||||||
max-width: 100vw;
|
|
||||||
max-height: 100vh;
|
|
||||||
}
|
|
||||||
|
|
||||||
.selection-manager {
|
|
||||||
position: fixed;
|
|
||||||
left: 10px;
|
|
||||||
right: 10px;
|
|
||||||
bottom: 65px;
|
|
||||||
box-sizing: border-box;
|
|
||||||
max-height: 40vh;
|
|
||||||
display: flex;
|
|
||||||
flex-direction: column;
|
|
||||||
padding: 15px 10px;
|
|
||||||
background-color: #181721;
|
|
||||||
border-radius: 10px;
|
|
||||||
box-shadow: 0 0 5px #0008;
|
|
||||||
}
|
|
||||||
|
|
||||||
.selection-manager hr {
|
|
||||||
margin: 5px 0;
|
|
||||||
}
|
|
||||||
|
|
||||||
.selection-header {
|
|
||||||
display: flex;
|
|
||||||
justify-content: space-between;
|
|
||||||
}
|
|
||||||
|
|
||||||
.selection-header > * {
|
|
||||||
cursor: pointer;
|
|
||||||
}
|
|
||||||
|
|
||||||
#selection-edit-tags {
|
|
||||||
color: #4DC7ED;
|
|
||||||
}
|
|
||||||
|
|
||||||
#selection-add-to-pool {
|
|
||||||
color: #F5E872;
|
|
||||||
}
|
|
||||||
|
|
||||||
#selection-delete {
|
|
||||||
color: #DB6060;
|
|
||||||
}
|
|
||||||
|
|
||||||
.selection-tags {
|
|
||||||
max-height: 100%;
|
|
||||||
overflow-x: hidden;
|
|
||||||
overflow-y: scroll;
|
|
||||||
}
|
|
||||||
|
|
||||||
input[type="color"] {
|
|
||||||
width: 100%;
|
|
||||||
}
|
|
||||||
|
|
||||||
.tags-container, .filtering-operators, .filtering-tokens {
|
|
||||||
padding: 5px;
|
|
||||||
background-color: #212529;
|
|
||||||
border: 1px solid #495057;
|
|
||||||
border-radius: .375rem;
|
|
||||||
display: flex;
|
|
||||||
justify-content: space-between;
|
|
||||||
align-content: flex-start;
|
|
||||||
align-items: flex-start;
|
|
||||||
flex-wrap: wrap;
|
|
||||||
box-sizing: border-box;
|
|
||||||
}
|
|
||||||
|
|
||||||
.filtering-operators {
|
|
||||||
margin-bottom: 15px;
|
|
||||||
}
|
|
||||||
|
|
||||||
.tags-container, .filtering-tokens {
|
|
||||||
margin: 15px 0;
|
|
||||||
height: 200px;
|
|
||||||
overflow-x: hidden;
|
|
||||||
overflow-y: scroll;
|
|
||||||
}
|
|
||||||
|
|
||||||
.tags-container:after, .filtering-tokens:after {
|
|
||||||
content: "";
|
|
||||||
flex: auto;
|
|
||||||
}
|
|
||||||
|
|
||||||
.tags-container-selected {
|
|
||||||
height: 100px;
|
|
||||||
}
|
|
||||||
|
|
||||||
#files-filter {
|
|
||||||
margin-bottom: 0;
|
|
||||||
height: 56px;
|
|
||||||
}
|
|
||||||
|
|
||||||
.viewer-wrapper {
|
|
||||||
position: absolute;
|
|
||||||
left: 0;
|
|
||||||
top: 0;
|
|
||||||
right: 0;
|
|
||||||
bottom: 0;
|
|
||||||
background-color: #000a;
|
|
||||||
display: flex;
|
|
||||||
justify-content: center;
|
|
||||||
/* overflow-y: scroll;*/
|
|
||||||
}
|
|
||||||
|
|
||||||
.viewer-nav {
|
|
||||||
position: absolute;
|
|
||||||
top: 0;
|
|
||||||
bottom: 0;
|
|
||||||
display: flex;
|
|
||||||
justify-content: center;
|
|
||||||
align-items: center;
|
|
||||||
cursor: pointer;
|
|
||||||
}
|
|
||||||
|
|
||||||
.viewer-nav:hover {
|
|
||||||
background-color: #b4adff40;
|
|
||||||
}
|
|
||||||
|
|
||||||
.viewer-nav-prev {
|
|
||||||
left: 0;
|
|
||||||
right: 80vw;
|
|
||||||
}
|
|
||||||
|
|
||||||
.viewer-nav-next {
|
|
||||||
left: 80vw;
|
|
||||||
right: 0;
|
|
||||||
}
|
|
||||||
|
|
||||||
.viewer-nav-close {
|
|
||||||
left: 0;
|
|
||||||
right: 0;
|
|
||||||
bottom: unset;
|
|
||||||
height: 15vh;
|
|
||||||
}
|
|
||||||
|
|
||||||
.viewer-nav-icon {
|
|
||||||
width: 20px;
|
|
||||||
height: 32px;
|
|
||||||
}
|
|
||||||
|
|
||||||
.viewer-nav-close > .viewer-nav-icon {
|
|
||||||
width: 16px;
|
|
||||||
height: 16px;
|
|
||||||
}
|
|
||||||
|
|
||||||
#viewer {
|
|
||||||
width: 100%;
|
|
||||||
height: 100%;
|
|
||||||
max-width: 100%;
|
|
||||||
}
|
|
||||||
|
|
||||||
.sessions-wrapper {
|
|
||||||
padding: 14px;
|
|
||||||
background-color: #111118;
|
|
||||||
border-radius: 10px;
|
|
||||||
}
|
|
||||||
|
|
||||||
.btn-terminate {
|
|
||||||
height: 20px;
|
|
||||||
cursor: pointer;
|
|
||||||
}
|
|
||||||
|
|
||||||
footer {
|
|
||||||
display: flex;
|
|
||||||
justify-content: space-between;
|
|
||||||
align-items: center;
|
|
||||||
background-color: #0007;
|
|
||||||
}
|
|
||||||
|
|
||||||
.nav {
|
|
||||||
display: flex;
|
|
||||||
justify-content: center;
|
|
||||||
align-items: center;
|
|
||||||
height: 40px;
|
|
||||||
width: 18vw;
|
|
||||||
background: transparent;
|
|
||||||
border: 0;
|
|
||||||
border-radius: 10px;
|
|
||||||
outline: 0;
|
|
||||||
}
|
|
||||||
|
|
||||||
.nav.curr, .nav:hover {
|
|
||||||
background-color: #343249;
|
|
||||||
}
|
|
||||||
|
|
||||||
.navicon {
|
|
||||||
display: block;
|
|
||||||
height: 30px;
|
|
||||||
}
|
|
||||||
|
|
||||||
#loader {
|
|
||||||
position: fixed;
|
|
||||||
left: 0;
|
|
||||||
top: 0;
|
|
||||||
right: 0;
|
|
||||||
bottom: 0;
|
|
||||||
display: flex;
|
|
||||||
justify-content: center;
|
|
||||||
align-items: center;
|
|
||||||
background-color: #000a;
|
|
||||||
z-index: 9999;
|
|
||||||
}
|
|
||||||
|
|
||||||
.loader-wrapper {
|
|
||||||
padding: 15px;
|
|
||||||
border-radius: 12px;
|
|
||||||
background-color: white;
|
|
||||||
}
|
|
||||||
|
|
||||||
.loader-img {
|
|
||||||
max-width: 20vw;
|
|
||||||
max-height: 20vh;
|
|
||||||
}
|
|
||||||
|
Before Width: | Height: | Size: 10 KiB |
|
Before Width: | Height: | Size: 15 KiB |
|
Before Width: | Height: | Size: 2.6 KiB |
|
Before Width: | Height: | Size: 3.4 KiB |
|
Before Width: | Height: | Size: 5.0 KiB |
|
Before Width: | Height: | Size: 6.6 KiB |
|
Before Width: | Height: | Size: 7.8 KiB |
|
Before Width: | Height: | Size: 8.3 KiB |
|
Before Width: | Height: | Size: 10 KiB |
|
Before Width: | Height: | Size: 11 KiB |
|
Before Width: | Height: | Size: 15 KiB |
|
Before Width: | Height: | Size: 3.9 KiB |
|
Before Width: | Height: | Size: 4.2 KiB |
|
Before Width: | Height: | Size: 5.0 KiB |
|
Before Width: | Height: | Size: 5.2 KiB |
|
Before Width: | Height: | Size: 16 KiB |
|
Before Width: | Height: | Size: 16 KiB |
|
Before Width: | Height: | Size: 1.6 KiB |
|
Before Width: | Height: | Size: 2.4 KiB |