feat: spotify-only matching + optional local-LLM gate + google→bandcamp fallback (closes task/drop-musicbrainz-google-fallback, closes task/ollama-classifier)

Source: human-feedback note 01KVDYV3A35B57N15CRBQDXPCH (project/reddit-spotify-linker,
2026-06-18) — "remove the musicbrainz integration and just use spotify"; a google search
for "<name> bandcamp" fallback; and "would a local llm be better able to categorise these
... include instructions for running something suitable via ollama (16GB M2)".

- Remove MusicBrainz entirely (viaMusicBrainz + url-rels + rate-gate; lib relUrls/mbResult).
  Spotify is now the only music API; it both gates by exact-name match and supplies the
  direct artist link.
- Optional ollama LLM gate (OLLAMA_URL enables it, OLLAMA_MODEL default qwen2.5:3b) called
  over plain HTTP -> stays zero npm-dep. It classifies which candidates are real
  artists/albums (the matching-quality lever). Pure prompt/parse logic in new llm.js
  (unit-tested in llm.test.js); the fetch + resolveAll gate wiring live in resolver.js.
- A confirmed name links to its direct Spotify page when available, else a Google search for
  "<name> bandcamp". New "google" primary platform rendered by the extension (styles.css
  blue accent + platform-aware link title in content.js).
- README: drop MusicBrainz; document the two gates + a runnable local-LLM setup for a 16GB
  MacBook M2. AGENTS.md: flip the old "no ML/NER" standing decision per the human override.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
paul
2026-06-19 12:50:07 +00:00
co-authored by Claude Opus 4.8
parent b97c82d92e
commit ef57761859
10 changed files with 343 additions and 249 deletions
+14 -5
View File
@@ -8,7 +8,10 @@ is tracked in the **towl** work-item store under project **`reddit-spotify-linke
- `service/` — zero-dependency Node resolver (Node 18+, built-in `fetch`/`http`).
`resolver.js` = the HTTP server; `lib.js` = pure, side-effect-free logic (the
artist-match precision lever) so it's unit-testable; `lib.test.js` = `node:test`.
artist-match precision lever) so it's unit-testable; `llm.js` = pure ollama
prompt/parse helpers (the optional local-LLM gate, HTTP-only — no in-process
model runtime); `lib.test.js` + `llm.test.js` = `node:test`. The only music API
is Spotify (MusicBrainz removed).
- `extension/` — Firefox **MV2** content script (`content.js` + `manifest.json` +
`styles.css`). No build step.
- `scripts/check.sh` — the local check gate.
@@ -56,7 +59,13 @@ fixes small things directly.
- Firefox **MV2** stays — Mozilla has no deprecation timeline; do not migrate to MV3
speculatively.
- **No ML/NER** in the resolver — keep it zero-dep; improve detection with cheap
heuristics + the API exact-match filter.
- Detection precision lever: a candidate links **only** when a music API returns an
artist whose name matches it case-insensitively.
- **Optional local LLM is the recommended better-matching path** (human override,
feedback 2026-06-18 — supersedes the old "no ML/NER" rule). The resolver may call
ollama over HTTP (`OLLAMA_URL` enables it, `OLLAMA_MODEL` defaults `qwen2.5:3b`)
to classify which candidates are real artists/albums. It stays **zero-NPM-dep**:
HTTP only, no in-process model runtime. The heuristic + Spotify exact-match is the
zero-dep default when no LLM is configured.
- Detection precision lever (zero-dep default): a candidate links **only** when
Spotify returns an artist whose name matches it case-insensitively. A confirmed
name links to its Spotify artist page if available, else a "<name> bandcamp"
Google fallback.
+63 -23
View File
@@ -8,15 +8,19 @@ list of clickable artists instead of names you have to copy-paste into search.
The hard part is *"which words are artists and which aren't?"*. Rather than guess
with client-side rules, the extension hands candidate names to a tiny local
**resolver service** that checks each one against a real music database and only
links the names that actually match an artist.
**resolver service** that confirms each one and only links the names that are
really artists. The resolver confirms names two ways (you enable either or both):
**Spotify** (exact-name match → direct artist pages) and an optional **local LLM**
via ollama (classifies which candidates are artists/albums). A confirmed name
links to its Spotify artist page when available, otherwise to a Google search for
"<name> bandcamp".
```
reddit page content script resolver service
─────────── ────────────── ────────────────
comment text ──extract──> candidate names ──POST /resolve──> Spotify API (direct links)
or
<a> links <──rewrite── confirmed links <───────────────── MusicBrainz (zero-setup)
comment text ──extract──> candidate names ──POST /resolve──> Spotify API (exact-match + direct links)
and/or
<a> links <──rewrite── confirmed links <───────────────── local LLM (ollama, optional)
```
## Layout
@@ -42,18 +46,18 @@ there are no dependencies.
```sh
cd service
node resolver.js
# resolver listening on http://localhost:8787 (provider: musicbrainz)
# resolver listening on http://localhost:8787 (spotify: off, llm: off)
```
That's it. With no configuration the resolver validates names against
**MusicBrainz** (no account required). When MusicBrainz knows an artist's official
links (its `url-relations`), you get **direct** Spotify and Bandcamp artist pages;
otherwise it falls back to Spotify/Bandcamp **search** URLs.
For anything to link you need **at least one** of two gates configured —
**Spotify credentials** or the **local LLM** (or both). With neither, the resolver
warns at startup and nothing links.
### Optional: direct Spotify links
### Spotify credentials (one of the two gates)
For **direct artist links** (instead of Spotify search), give the service Spotify
API credentials:
Spotify does two jobs: it **gates** a name (a candidate links only when Spotify has
that artist by exact name match) and supplies the **direct** artist-page link. To
enable it:
1. Create a free app at <https://developer.spotify.com/dashboard> and copy its
**Client ID** and **Client Secret**.
@@ -63,10 +67,11 @@ API credentials:
SPOTIFY_CLIENT_ID=your_id
SPOTIFY_CLIENT_SECRET=your_secret
```
3. Restart the service. It will report `provider: spotify` on startup.
3. Restart the service. Startup reports `(spotify: on, llm: off)`.
(Bandcamp has no public API; direct Bandcamp links come from MusicBrainz
`url-relations` when present, else a Bandcamp search URL.)
With Spotify alone, names that aren't on Spotify don't link — precise, but it
misses real artists Spotify doesn't carry. The local LLM gate (below) catches
those.
Quick check that it works:
@@ -77,6 +82,37 @@ curl -s localhost:8787/resolve \
# {"Cerebral Bore":{...},"Devourment":{...},"Notarealband Xyzzy":null}
```
A confirmed name links to its Spotify artist page when available, otherwise to a
Google search for "<name> bandcamp" (which surfaces the artist's Bandcamp page).
## Better matching with a local LLM (optional)
The optional LLM gate is the **matching-quality lever**: it classifies which
candidate strings are real artists/albums, so confirmed names that aren't on
Spotify still link (via the "<name> bandcamp" Google fallback). The resolver calls
ollama over **plain HTTP**, so it stays **zero npm dependencies**.
On a 16GB MacBook (M2):
1. Install ollama — <https://ollama.com/download> or `brew install ollama`.
2. Pull a small model: `ollama pull qwen2.5:3b` (~2GB; runs comfortably on a 16GB
M2; the first resolve call warms the model so it's a little slow once, then
fast).
3. In `service/.env`, point the resolver at ollama (and optionally override the
model):
```
OLLAMA_URL=http://localhost:11434
OLLAMA_MODEL=qwen2.5:3b
```
4. Restart the resolver. Startup reports `(spotify: off, llm: qwen2.5:3b)` (or
`(spotify: on, llm: qwen2.5:3b)` if you also set Spotify creds).
5. Confirm: `curl localhost:8787/health` should show `"llm":"qwen2.5:3b"`.
With both gates, the LLM decides which names are artists and Spotify supplies the
direct links; with the LLM alone, confirmed names get a Spotify direct link if one
exists, else the Google fallback.
## 2. Install the extension in Firefox
The extension is unpacked / unsigned, so load it as a **temporary add-on**:
@@ -104,7 +140,9 @@ with `xpinstall.signatures.required = false`.)
If nothing gets linked:
1. Confirm the resolver is running — `curl localhost:8787/health` should return
`{"ok":true,...}`.
`{"ok":true,"spotify":<bool>,"llm":<model|false>,"cached":<n>}`. If both
`spotify` and `llm` are off, nothing can link — configure at least one gate
above.
2. On the reddit thread, open the browser console (F12) and look for `[rsl]` logs: they
report the root element, candidate count, and resolver responses on each scan. A
`resolve failed` line means the background script can't reach the resolver (is it on
@@ -117,7 +155,7 @@ If nothing gets linked:
## Configuration
The resolver URL is the constant `RESOLVER_URL` at the top of
`extension/content.js` (default `http://localhost:8787`). If you change the host
`extension/background.js` (default `http://localhost:8787`). If you change the host
or port, update **both** that constant **and** the matching entry in
`extension/manifest.json` → `permissions` (Firefox needs the host listed there to
let the content script reach it).
@@ -128,12 +166,14 @@ let the content script reach it).
lowercase names right after "check out / by / fan of / recommend …". Names split across
inline formatting (e.g. **bold** mid-name) can still be missed. The resolver's
exact-name match keeps false *positives* low; the trade-off is some false *negatives*.
- **Links are direct when MusicBrainz has the artist's `url-relations`** (official
Bandcamp / Spotify pages); otherwise they fall back to a search URL. (Each
confirmed artist costs one extra throttled MusicBrainz lookup, cached after.)
- **Links are direct to Spotify when Spotify has the artist**; otherwise a confirmed
name links to a Google search for "<name> bandcamp" (which surfaces the artist's
Bandcamp page).
- **Without Spotify creds or the LLM, nothing links** — at least one gate must be
configured.
- The optional LLM adds a little latency on the **first** call (ollama warms the
model), then is fast.
- The resolver caches results in memory; it resets when you restart it.
- Without Spotify credentials the MusicBrainz path is rate-limited to ~1 lookup/
second, so the first scan of a large thread fills in gradually (cached after).
- Built and tested as a Firefox MV2 extension.
## License
+9 -5
View File
@@ -46,17 +46,21 @@
}
// ---- link building / rewriting --------------------------------------------
// Per the user's "if on spotify, link that, else bandcamp" rule: one link to the
// primary platform, colour-coded so you can tell which at a glance.
// Per the user's "if on spotify, link that, else a Bandcamp Google-search fallback"
// rule: one link to the primary platform, colour-coded so you can tell which at a glance.
function makeLink(label, links) {
const primary = links.primary || { platform: "spotify", url: links.spotify };
const name = links.name || label;
const platform = primary.platform;
const a = document.createElement("a");
a.className = `rsl-link rsl-${primary.platform}`;
a.className = `rsl-link rsl-${platform}`;
a.textContent = label;
a.href = primary.url;
a.target = "_blank";
a.rel = "noopener noreferrer";
a.title = `Open on ${primary.platform === "spotify" ? "Spotify" : "Bandcamp"}`;
a.title = platform === "spotify"
? `Open ${name} on Spotify`
: `Find ${name} on Bandcamp (Google search)`;
return a;
}
@@ -80,7 +84,7 @@
}
// ---- resolver client ------------------------------------------------------
const resolved = new Map(); // candidate string -> { spotify, bandcamp } | null
const resolved = new Map(); // candidate string -> { name, spotify?, google?, primary } | null
// The network call goes through the background script — a page-context fetch to
// http://localhost is blocked by reddit's CSP / mixed-content rules.
+3 -3
View File
@@ -1,5 +1,5 @@
/* Injected links — kept subtle so reddit comments stay readable. One link per artist,
coloured by platform: green = Spotify, teal = Bandcamp. */
coloured by platform: green = Spotify, blue = Google (bandcamp-search fallback). */
.rsl-link {
text-decoration: underline dotted;
text-underline-offset: 2px;
@@ -8,8 +8,8 @@
.rsl-spotify {
text-decoration-color: #1db954 !important; /* spotify green */
}
.rsl-bandcamp {
text-decoration-color: #629aa9 !important; /* bandcamp teal */
.rsl-google {
text-decoration-color: #4a72d0 !important; /* google-ish blue */
}
.rsl-link:hover {
text-decoration-style: solid !important;
+13 -8
View File
@@ -1,13 +1,18 @@
# Copy to `.env` (it is gitignored). All values are OPTIONAL.
# Copy to `.env` (it is gitignored).
#
# Without Spotify credentials the resolver validates names against MusicBrainz and links
# to Spotify/Bandcamp SEARCH pages — works with zero setup.
#
# With credentials it uses the Spotify Web API for DIRECT artist links.
# Create a (free) app at https://developer.spotify.com/dashboard to get these.
# Spotify Web API (client-credentials) — gives DIRECT artist links and, on its own, gates
# candidates by exact-name match (a name links only when Spotify has that artist). Create a
# free app at https://developer.spotify.com/dashboard to get these.
SPOTIFY_CLIENT_ID=
SPOTIFY_CLIENT_SECRET=
# Port the resolver listens on (default 8787). If you change it, also update
# RESOLVER_URL in extension/content.js and the host in extension/manifest.json.
# Optional local LLM (via ollama) for BETTER matching it decides which candidate substrings
# are real artists/albums. Set OLLAMA_URL to enable it; still zero npm deps (plain HTTP).
# Confirmed names get a direct Spotify link when available, else a Google->Bandcamp fallback.
# ollama pull qwen2.5:3b # ~2GB, runs comfortably on a 16GB MacBook M2
# OLLAMA_URL=http://localhost:11434
# OLLAMA_MODEL=qwen2.5:3b
# Port the resolver listens on (default 8787). If you change it, also update RESOLVER_URL in
# extension/background.js and the host in extension/manifest.json permissions.
PORT=8787
+18 -57
View File
@@ -4,44 +4,22 @@
export const norm = (s) => String(s).normalize("NFKC").trim().toLowerCase();
export const bandcampSearch = (n) => `https://bandcamp.com/search?q=${encodeURIComponent(n)}&item_type=b`;
export const spotifySearch = (n) => `https://open.spotify.com/search/${encodeURIComponent(n)}`;
// Fallback link when we have no direct Spotify page: a Google search biased at Bandcamp,
// which surfaces the artist's Bandcamp page (when it exists) plus general results.
export const googleSearch = (n) => `https://www.google.com/search?q=${encodeURIComponent(`${n} bandcamp`)}`;
// The precision lever: return the result whose name matches the candidate
// case-insensitively, else null. `minScore` (MusicBrainz) additionally requires a
// confidence score. Tolerates null/garbage items.
export function pickArtist(name, items, { minScore = null } = {}) {
// The precision lever: return the item whose name matches the candidate
// case-insensitively, else null. Tolerates null/garbage items.
export function pickArtist(name, items) {
const want = norm(name);
return (
(items ?? []).find(
(a) => a && norm(a.name) === want && (minScore == null || (a.score ?? 0) >= minScore),
) ?? null
);
return (items ?? []).find((a) => a && norm(a.name) === want) ?? null;
}
// A "/search" url means we don't have a direct artist page (just a search fallback).
export const isSearchUrl = (url) => typeof url === "string" && url.includes("/search");
// Choose the single best link to show: prefer Spotify when the artist is actually ON
// Spotify (a direct, non-search url); else a direct Bandcamp; else the Spotify search.
export function primaryOf({ spotify, bandcamp }) {
if (spotify && !isSearchUrl(spotify)) return { platform: "spotify", url: spotify };
if (bandcamp && !isSearchUrl(bandcamp)) return { platform: "bandcamp", url: bandcamp };
return { platform: "spotify", url: spotify };
}
// Link object for a confirmed Spotify hit (direct artist url when present, else search).
export function spotifyResult(hit) {
const r = {
name: hit.name,
spotify: hit.external_urls?.spotify ?? spotifySearch(hit.name),
bandcamp: bandcampSearch(hit.name),
};
return { ...r, primary: primaryOf(r) };
}
// True for http/https URLs only — reject javascript:, ftp:, garbage. These values flow
// into an injected link's href, so validate the third-party MB data defensively.
// True for http/https URLs only — reject javascript:, ftp:, garbage. The Spotify
// external_url flows into an injected link's href, so validate it defensively.
export function isHttpUrl(u) {
try {
const p = new URL(u).protocol;
@@ -51,31 +29,14 @@ export function isHttpUrl(u) {
}
}
// Extract direct links from a MusicBrainz artist's url-relations (zero-cred: MB stores
// official Bandcamp / streaming URLs). Accepts only http(s) and matches by parsed hostname
// (not substring, so "evilbandcamp.com" / "javascript:…bandcamp.com…" don't slip through).
// Either field may be null.
export function relUrls(relations) {
const urls = (relations ?? []).map((r) => r && r.url && r.url.resource).filter(isHttpUrl);
const byHost = (host) =>
urls.find((u) => {
try {
const h = new URL(u).hostname;
return h === host || h.endsWith("." + host);
} catch {
return false;
// Build the single link to show for a confirmed name. Prefer the direct Spotify artist
// url (validated http(s), not a /search url); otherwise fall back to a Google→Bandcamp
// search. `primary` is what the extension renders ({ platform, url }).
export function linkResult(name, spotifyUrl) {
const direct = spotifyUrl && isHttpUrl(spotifyUrl) && !isSearchUrl(spotifyUrl) ? spotifyUrl : null;
if (direct) {
return { name, spotify: direct, primary: { platform: "spotify", url: direct } };
}
}) ?? null;
return { bandcamp: byHost("bandcamp.com"), spotify: byHost("open.spotify.com") };
}
// Link object for a confirmed MusicBrainz hit: prefer the direct url-rel links, fall back
// to search urls when MB has no relation for that platform.
export function mbResult(hit, links) {
const r = {
name: hit.name,
spotify: links?.spotify ?? spotifySearch(hit.name),
bandcamp: links?.bandcamp ?? bandcampSearch(hit.name),
};
return { ...r, primary: primaryOf(r) };
const url = googleSearch(name);
return { name, google: url, primary: { platform: "google", url } };
}
+29 -86
View File
@@ -1,7 +1,7 @@
// Unit tests for the resolver's pure logic (run with `node --test`). Zero-dep.
import { test } from "node:test";
import assert from "node:assert/strict";
import { norm, bandcampSearch, spotifySearch, pickArtist, spotifyResult, relUrls, mbResult, primaryOf, isSearchUrl, isHttpUrl } from "./lib.js";
import { norm, googleSearch, pickArtist, linkResult, isSearchUrl, isHttpUrl } from "./lib.js";
test("norm lowercases, trims, NFKC-normalizes", () => {
assert.equal(norm(" Cerebral Bore "), "cerebral bore");
@@ -15,78 +15,44 @@ test("pickArtist matches case-insensitively, rejects non-matches", () => {
assert.equal(pickArtist("Notarealband", items), null);
});
test("pickArtist enforces minScore when given (MusicBrainz path)", () => {
const items = [{ name: "Devourment", score: 100 }, { name: "Devourment", score: 50 }];
assert.equal(pickArtist("Devourment", items, { minScore: 90 })?.score, 100);
assert.equal(pickArtist("Devourment", [{ name: "Devourment", score: 50 }], { minScore: 90 }), null);
});
test("pickArtist tolerates empty / garbage input", () => {
assert.equal(pickArtist("x", null), null);
assert.equal(pickArtist("x", []), null);
assert.equal(pickArtist("x", [null, { name: "y" }]), null);
});
test("spotifyResult prefers the direct url, falls back to search", () => {
const direct = spotifyResult({
name: "Devourment",
external_urls: { spotify: "https://open.spotify.com/artist/abc" },
});
assert.equal(direct.spotify, "https://open.spotify.com/artist/abc");
assert.match(direct.bandcamp, /^https:\/\/bandcamp\.com\/search\?q=Devourment/);
assert.deepEqual(direct.primary, { platform: "spotify", url: "https://open.spotify.com/artist/abc" });
const noUrl = spotifyResult({ name: "Devourment" });
assert.equal(noUrl.spotify, "https://open.spotify.com/search/Devourment");
assert.equal(noUrl.primary.platform, "spotify"); // search fallback still points at spotify
test("googleSearch points at a Google query for '<name> bandcamp', url-encoded", () => {
const u = googleSearch("Cerebral Bore");
assert.match(u, /^https:\/\/www\.google\.com\/search\?q=/);
assert.ok(u.includes("Cerebral%20Bore%20bandcamp"));
assert.ok(googleSearch("A&B").includes("A%26B"));
});
test("primaryOf / isSearchUrl: prefer direct spotify, else direct bandcamp, else spotify search", () => {
test("linkResult prefers a direct Spotify url, primary = spotify", () => {
const r = linkResult("Devourment", "https://open.spotify.com/artist/abc");
assert.equal(r.spotify, "https://open.spotify.com/artist/abc");
assert.deepEqual(r.primary, { platform: "spotify", url: "https://open.spotify.com/artist/abc" });
assert.equal(r.google, undefined);
});
test("linkResult falls back to Google->Bandcamp when there is no direct Spotify url", () => {
const r = linkResult("Some Local Band", null);
assert.equal(r.spotify, undefined);
assert.equal(r.primary.platform, "google");
assert.match(r.primary.url, /^https:\/\/www\.google\.com\/search\?q=/);
assert.equal(r.google, r.primary.url);
});
test("linkResult rejects non-http(s) / search Spotify urls and falls back to Google", () => {
for (const bad of ["javascript:alert(1)", "ftp://x/y", "https://open.spotify.com/search/X", "not a url"]) {
const r = linkResult("X", bad);
assert.equal(r.primary.platform, "google", `expected fallback for ${bad}`);
}
});
test("isSearchUrl detects /search urls", () => {
assert.equal(isSearchUrl("https://open.spotify.com/search/X"), true);
assert.equal(isSearchUrl("https://open.spotify.com/artist/abc"), false);
assert.deepEqual(
primaryOf({ spotify: "https://open.spotify.com/artist/abc", bandcamp: "https://bandcamp.com/search?q=X&item_type=b" }),
{ platform: "spotify", url: "https://open.spotify.com/artist/abc" },
);
assert.deepEqual(
primaryOf({ spotify: "https://open.spotify.com/search/X", bandcamp: "https://x.bandcamp.com/" }),
{ platform: "bandcamp", url: "https://x.bandcamp.com/" },
);
assert.deepEqual(
primaryOf({ spotify: "https://open.spotify.com/search/X", bandcamp: "https://bandcamp.com/search?q=X&item_type=b" }),
{ platform: "spotify", url: "https://open.spotify.com/search/X" },
);
});
test("relUrls picks direct bandcamp + spotify from MB relations, tolerates junk", () => {
const relations = [
{ url: { resource: "https://cattledecapitation.bandcamp.com/" } },
{ url: { resource: "https://open.spotify.com/artist/67ZMMtA88DDO0gTuRrzGjn" } },
{ url: { resource: "https://en.wikipedia.org/wiki/Cattle_Decapitation" } },
null,
{ url: null },
];
const links = relUrls(relations);
assert.equal(links.bandcamp, "https://cattledecapitation.bandcamp.com/");
assert.equal(links.spotify, "https://open.spotify.com/artist/67ZMMtA88DDO0gTuRrzGjn");
});
test("relUrls returns nulls when relations absent/empty", () => {
assert.deepEqual(relUrls(null), { bandcamp: null, spotify: null });
assert.deepEqual(relUrls([]), { bandcamp: null, spotify: null });
});
test("relUrls rejects non-http(s) and look-alike hosts (hardening)", () => {
const relations = [
{ url: { resource: "javascript:alert('bandcamp.com')" } }, // not http(s)
{ url: { resource: "ftp://files.bandcamp.com/x" } }, // not http(s)
{ url: { resource: "https://evilbandcamp.com/" } }, // host doesn't end in .bandcamp.com
{ url: { resource: "https://x.bandcamp.com/" } }, // valid subdomain
];
const links = relUrls(relations);
assert.equal(links.bandcamp, "https://x.bandcamp.com/");
assert.equal(links.spotify, null);
});
test("isHttpUrl accepts http/https only", () => {
@@ -97,26 +63,3 @@ test("isHttpUrl accepts http/https only", () => {
assert.equal(isHttpUrl("not a url"), false);
assert.equal(isHttpUrl(null), false);
});
test("mbResult prefers direct links, falls back to search per-platform", () => {
const direct = mbResult(
{ name: "Cattle Decapitation" },
{ bandcamp: "https://cattledecapitation.bandcamp.com/", spotify: "https://open.spotify.com/artist/x" },
);
assert.equal(direct.bandcamp, "https://cattledecapitation.bandcamp.com/");
assert.equal(direct.spotify, "https://open.spotify.com/artist/x");
assert.deepEqual(direct.primary, { platform: "spotify", url: "https://open.spotify.com/artist/x" });
const fallback = mbResult({ name: "Cerebral Bore" }, null);
assert.equal(fallback.spotify, "https://open.spotify.com/search/Cerebral%20Bore");
assert.equal(fallback.bandcamp, "https://bandcamp.com/search?q=Cerebral%20Bore&item_type=b");
const partial = mbResult({ name: "X" }, { bandcamp: null, spotify: "https://open.spotify.com/artist/y" });
assert.equal(partial.spotify, "https://open.spotify.com/artist/y");
assert.match(partial.bandcamp, /bandcamp\.com\/search/);
});
test("link builders url-encode", () => {
assert.equal(spotifySearch("A B"), "https://open.spotify.com/search/A%20B");
assert.ok(bandcampSearch("A&B").includes("A%26B"));
});
+51
View File
@@ -0,0 +1,51 @@
// Pure helpers for the OPTIONAL local-LLM (ollama) classifier. The HTTP call lives in
// resolver.js; the prompt builder + reply parser live here so they unit-test with
// `node --test` (no network). The LLM answers one question: "which of these candidate
// substrings are music artists/albums?" — the precision gate when it is configured.
import { norm } from "./lib.js";
// Small, fast, runs comfortably on a 16GB M2 (3.1B params, Q4 ~2GB). Override with OLLAMA_MODEL.
export const DEFAULT_MODEL = "qwen2.5:3b";
// Chat messages for an ollama /api/chat call (used with stream:false, format:"json").
export function buildClassifyMessages(candidates) {
const system =
"You are a music-name classifier for a tool that links artist/album mentions in Reddit " +
"music discussions. You are given a JSON array of candidate text snippets pulled from reddit " +
"comments. For EACH candidate decide whether it names a music ARTIST/BAND, a music " +
"ALBUM/release, or NEITHER. The snippets come from music threads, so prefer the music reading " +
"when a candidate is a plausible band/album even if it is also an ordinary word. Mark ordinary " +
"words, sentence fragments, generic phrases, place names, and non-music proper nouns as \"none\". " +
'Reply with ONLY a JSON object of the form {"results":[{"name":<candidate verbatim>,' +
'"kind":"artist"|"album"|"none"}]}. Use each candidate string VERBATIM as its name.';
const user = JSON.stringify({ candidates });
return [
{ role: "system", content: system },
{ role: "user", content: user },
];
}
// Parse the model's JSON reply into a Set of norm(name) the model marked artist/album.
// Defensive: only names that were actually in the candidate list count (guards against the
// model inventing names); an unparseable reply yields an empty Set, and the caller then
// falls back to the Spotify-exact-match gate.
export function parseClassification(text, candidates) {
const allowed = new Set(candidates.map(norm));
const confirmed = new Set();
let obj;
try {
obj = JSON.parse(text);
} catch {
return confirmed;
}
const rows = Array.isArray(obj?.results) ? obj.results : Array.isArray(obj) ? obj : [];
for (const row of rows) {
if (!row || typeof row.name !== "string") continue;
const k = norm(row.name);
if (!allowed.has(k)) continue;
const kind = String(row.kind ?? "").toLowerCase();
if (kind === "artist" || kind === "album") confirmed.add(k);
}
return confirmed;
}
+55
View File
@@ -0,0 +1,55 @@
// Unit tests for the optional LLM classifier's pure logic (run with `node --test`). Zero-dep.
import { test } from "node:test";
import assert from "node:assert/strict";
import { buildClassifyMessages, parseClassification, DEFAULT_MODEL } from "./llm.js";
test("DEFAULT_MODEL is a small ollama model", () => {
assert.equal(DEFAULT_MODEL, "qwen2.5:3b");
});
test("buildClassifyMessages embeds the candidate list as JSON for the user turn", () => {
const msgs = buildClassifyMessages(["Tool", "the best"]);
assert.equal(msgs.length, 2);
assert.equal(msgs[0].role, "system");
assert.match(msgs[0].content, /artist/i);
assert.equal(msgs[1].role, "user");
assert.deepEqual(JSON.parse(msgs[1].content), { candidates: ["Tool", "the best"] });
});
test("parseClassification keeps only artist/album candidates that were in the list", () => {
const candidates = ["Cattle Decapitation", "Devourment", "the best part"];
const reply = JSON.stringify({
results: [
{ name: "Cattle Decapitation", kind: "artist" },
{ name: "Devourment", kind: "ARTIST" }, // case-insensitive kind
{ name: "the best part", kind: "none" },
],
});
const got = parseClassification(reply, candidates);
assert.ok(got.has("cattle decapitation"));
assert.ok(got.has("devourment"));
assert.ok(!got.has("the best part"));
});
test("parseClassification accepts album kind and a bare array reply", () => {
const got = parseClassification(
JSON.stringify([{ name: "Reek of Putrefaction", kind: "album" }]),
["Reek of Putrefaction"],
);
assert.ok(got.has("reek of putrefaction"));
});
test("parseClassification ignores hallucinated names not in the candidate list", () => {
const got = parseClassification(
JSON.stringify({ results: [{ name: "Some Band I Invented", kind: "artist" }] }),
["Real Candidate"],
);
assert.equal(got.size, 0);
});
test("parseClassification returns an empty set on unparseable / junk replies", () => {
assert.equal(parseClassification("not json at all", ["X"]).size, 0);
assert.equal(parseClassification("", ["X"]).size, 0);
assert.equal(parseClassification(JSON.stringify({ nope: true }), ["X"]).size, 0);
assert.equal(parseClassification(JSON.stringify({ results: [null, { kind: "artist" }] }), ["X"]).size, 0);
});
+87 -61
View File
@@ -1,16 +1,20 @@
// Tiny resolver service: candidate names -> artist links, for CONFIRMED artists only.
//
// Zero npm deps; Node 18+ (built-in fetch). The "which words are real artists" problem
// is delegated to a music API rather than guessed client-side:
// * with SPOTIFY_CLIENT_ID/SECRET set -> Spotify Web API (client-credentials) for
// DIRECT artist links;
// * without creds -> MusicBrainz validation + Spotify/Bandcamp SEARCH links (zero setup).
// Either way a candidate only resolves when an artist's name matches it case-insensitively
// (the precision lever). The pure matching/link logic lives in lib.js (unit-tested).
// Zero npm deps; Node 18+ (built-in fetch). The "which words are real artists" problem is
// not guessed client-side — it is gated one of two ways:
// * SPOTIFY_CLIENT_ID/SECRET set -> a candidate resolves only when Spotify has an artist
// whose name matches it case-insensitively (the exact-match precision lever), linking to
// the DIRECT Spotify artist page.
// * OLLAMA_URL set -> a local LLM (via ollama HTTP, still zero npm deps) classifies which
// candidates are artists/albums; confirmed names get a Spotify direct link when available,
// else a Google->Bandcamp search fallback. This is the better-matching path.
// The two combine: with both set, the LLM gates and Spotify supplies direct links.
// The pure matching/link/prompt logic lives in lib.js + llm.js (unit-tested).
import { createServer } from "node:http";
import { readFileSync } from "node:fs";
import { norm, pickArtist, spotifyResult, mbResult, relUrls } from "./lib.js";
import { norm, pickArtist, linkResult } from "./lib.js";
import { buildClassifyMessages, parseClassification, DEFAULT_MODEL } from "./llm.js";
// --- minimal .env loader (no dependency) ------------------------------------
try {
@@ -25,12 +29,16 @@ const PORT = Number(process.env.PORT) || 8787;
const SPOTIFY_ID = process.env.SPOTIFY_CLIENT_ID;
const SPOTIFY_SECRET = process.env.SPOTIFY_CLIENT_SECRET;
const USE_SPOTIFY = Boolean(SPOTIFY_ID && SPOTIFY_SECRET);
const UA = "reddit-spotify-linker/0.1 (https://git.yapplesauce.com/paul/linker)";
const OLLAMA_URL = (process.env.OLLAMA_URL || "").replace(/\/$/, ""); // presence enables the LLM gate
const OLLAMA_MODEL = process.env.OLLAMA_MODEL || DEFAULT_MODEL;
const USE_LLM = Boolean(OLLAMA_URL);
const MAX_CANDIDATES = 100;
const cache = new Map(); // norm(name) -> { spotify, bandcamp, name } | null (negatives cached too)
// Cache only the network half (Spotify direct-url lookup); the final link object is rebuilt
// per request so a name's result never depends on a stale gate decision. norm(name) -> url|null
const spCache = new Map();
// --- Spotify (optional) -----------------------------------------------------
// --- Spotify (optional): direct-artist-link lookup --------------------------
let spToken = null;
let spTokenExp = 0;
async function spotifyToken() {
@@ -48,72 +56,85 @@ async function spotifyToken() {
return spToken;
}
async function viaSpotify(name) {
// The direct Spotify artist url for an exact name match, or null (no creds / no exact hit).
async function spotifyDirect(name) {
if (!USE_SPOTIFY) return null;
const token = await spotifyToken();
const url = `https://api.spotify.com/v1/search?type=artist&limit=5&q=${encodeURIComponent(name)}`;
const r = await fetch(url, { headers: { authorization: `Bearer ${token}` } });
if (!r.ok) throw new Error(`spotify search http ${r.status}`);
const j = await r.json();
const hit = pickArtist(name, j.artists?.items);
return hit ? spotifyResult(hit) : null;
return hit?.external_urls?.spotify ?? null;
}
// --- MusicBrainz (zero-cred path) -------------------------------------------
// MusicBrainz asks for <=1 request/second, so every request is serialized + spaced.
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
let mbGate = Promise.resolve();
let mbLast = 0;
function mbSchedule() {
const gate = mbGate.then(async () => {
const wait = 1100 - (Date.now() - mbLast);
if (wait > 0) await sleep(wait);
mbLast = Date.now();
});
mbGate = gate.catch(() => {}); // a failure never wedges the queue
return gate;
}
async function mbFetchJson(url) {
await mbSchedule();
const r = await fetch(url, { headers: { "user-agent": UA, accept: "application/json" } });
if (!r.ok) throw new Error(`musicbrainz http ${r.status}`);
return r.json();
}
async function viaMusicBrainz(name) {
const q = encodeURIComponent(`artist:"${name}"`);
const search = await mbFetchJson(`https://musicbrainz.org/ws/2/artist?fmt=json&limit=5&query=${q}`);
const hit = pickArtist(name, search.artists, { minScore: 90 });
if (!hit) return null;
// Enrich with MB url-relations -> DIRECT bandcamp/spotify links (best-effort).
let links = null;
try {
const detail = await mbFetchJson(`https://musicbrainz.org/ws/2/artist/${hit.id}?inc=url-rels&fmt=json`);
links = relUrls(detail.relations);
} catch (e) {
console.error(`mb url-rels "${name}": ${e.message}`); // enrichment is best-effort
}
return mbResult(hit, links);
}
// --- resolve one candidate (cached) -----------------------------------------
async function resolveOne(name) {
async function spotifyDirectCached(name) {
const key = norm(name);
if (!key) return null;
if (cache.has(key)) return cache.get(key);
if (spCache.has(key)) return spCache.get(key);
const url = await spotifyDirect(name);
spCache.set(key, url);
return url;
}
// --- LLM gate (optional): which candidates are artists/albums? --------------
async function ollamaClassify(candidates) {
const r = await fetch(`${OLLAMA_URL}/api/chat`, {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({
model: OLLAMA_MODEL,
messages: buildClassifyMessages(candidates),
stream: false,
format: "json",
options: { temperature: 0 },
}),
});
if (!r.ok) throw new Error(`ollama http ${r.status}`);
const j = await r.json();
return parseClassification(j.message?.content ?? "", candidates);
}
// --- resolve --------------------------------------------------------------
// allowFallback: when the gate has already confirmed this name is an artist/album, a missing
// Spotify direct link still yields a Google->Bandcamp fallback link (instead of null).
async function resolveOne(name, allowFallback) {
if (!norm(name)) return null;
try {
const result = USE_SPOTIFY ? await viaSpotify(name) : await viaMusicBrainz(name);
cache.set(key, result); // cache hits AND confirmed misses
return result;
const sp = await spotifyDirectCached(name);
if (sp) return linkResult(name, sp); // direct Spotify page
return allowFallback ? linkResult(name, null) : null; // google fallback, or unconfirmed
} catch (e) {
console.error(`resolve "${name}": ${e.message}`); // transient -> don't cache
console.error(`resolve "${name}": ${e.message}`); // transient -> caller sees null this time
return null;
}
}
async function resolveAll(candidates) {
const uniq = [...new Set(candidates.map((c) => String(c).trim()).filter(Boolean))].slice(0, MAX_CANDIDATES);
// Gate: which names may link? null => no LLM gate, the Spotify exact-match is the gate
// (only names actually on Spotify link). A Set => the LLM's verdict; names not in it are
// dropped, names in it may use the Google fallback when not on Spotify.
let confirmed = null;
if (USE_LLM && uniq.length) {
try {
confirmed = await ollamaClassify(uniq);
} catch (e) {
console.error(`llm classify: ${e.message}`); // LLM down -> fall back to the Spotify gate
confirmed = null;
}
}
const out = {};
await Promise.all(uniq.map(async (name) => { out[name] = await resolveOne(name); }));
await Promise.all(
uniq.map(async (name) => {
if (confirmed && !confirmed.has(norm(name))) {
out[name] = null; // LLM said this is not an artist/album
return;
}
out[name] = await resolveOne(name, confirmed != null);
}),
);
return out;
}
@@ -142,7 +163,7 @@ const server = createServer(async (req, res) => {
if (req.method === "OPTIONS") { res.writeHead(204); return res.end(); }
if (req.method === "GET" && req.url === "/health") {
return json(res, 200, { ok: true, provider: USE_SPOTIFY ? "spotify" : "musicbrainz", cached: cache.size });
return json(res, 200, { ok: true, spotify: USE_SPOTIFY, llm: USE_LLM ? OLLAMA_MODEL : false, cached: spCache.size });
}
if (req.method === "POST" && req.url === "/resolve") {
@@ -159,5 +180,10 @@ const server = createServer(async (req, res) => {
});
server.listen(PORT, () => {
console.log(`resolver listening on http://localhost:${PORT} (provider: ${USE_SPOTIFY ? "spotify" : "musicbrainz"})`);
const sp = USE_SPOTIFY ? "on" : "off";
const llm = USE_LLM ? OLLAMA_MODEL : "off";
console.log(`resolver listening on http://localhost:${PORT} (spotify: ${sp}, llm: ${llm})`);
if (!USE_SPOTIFY && !USE_LLM) {
console.warn(" ! no gate configured: set SPOTIFY_CLIENT_ID/SECRET and/or OLLAMA_URL, or nothing will link.");
}
});