From 13e70cbd58a28b7c1171457a5aa2a1f89ccdf314 Mon Sep 17 00:00:00 2001 From: Lawrence Hook Date: Mon, 2 Mar 2026 00:18:38 -0500 Subject: [PATCH] version 4.3.73 Squashed commit of the following: commit dbe3eae4052b32f3bb1e23067dc13cf81a80c7bf Author: Lawrence Hook Date: Mon Mar 2 00:13:35 2026 -0500 Add Stripe checkout hint to upgrade modal Co-Authored-By: Claude Opus 4.6 commit b4337277546084281b293dad7ca6e73c87e2b35b Author: Lawrence Hook Date: Sun Mar 1 23:48:01 2026 -0500 Add email validation, IP rate limiting, and fix SES parsing bug Add SES email validation with 2s timeout and fail-open behavior to send-magic-link. Add per-IP rate limiting with decrement on verify. Fix SES response parsing path (MailboxValidation.IsValid.ConfidenceVerdict). Add dedicated unit tests for validateEmail. Co-Authored-By: Claude Opus 4.6 commit 84b103e46bd58254b2d109ffd76072d3c22f3c17 Author: Lawrence Hook Date: Sun Mar 1 23:47:51 2026 -0500 Migrate storage layer from flat files to SQLite Replace file-per-record auth requests, file-per-hash rate limits, and JSON blob subscription cache with a single SQLite database via better-sqlite3. Consolidate duplicated email/IP rate limit code into shared helpers. Add missing IP rate limit pruning to cleanup interval. Co-Authored-By: Claude Opus 4.6 commit 84c2ea558633c2ac69eaab235867fa4e263f8754 Author: Lawrence Hook Date: Sun Mar 1 21:58:47 2026 -0500 update versions commit a4d5a97664286c78af069357fe05c843dbc9d7ea Merge: 53a18e7 2fe77fa Author: Lawrence Hook Date: Sun Mar 1 21:58:09 2026 -0500 Merge branch 'released' commit 53a18e7987ec332b280be0fe473336df332f8c6a Author: Lawrence Hook Date: Sun Mar 1 16:56:50 2026 -0500 Hide donate link for premium users instead of showing dead label Co-Authored-By: Claude Opus 4.6 commit e2732dee6fbe2aeac77a56e87281b01000a13d89 Author: Lawrence Hook Date: Sun Mar 1 16:50:33 2026 -0500 Fix fast-xml-parser vulnerability via npm override and update @aws-sdk/client-ses Co-Authored-By: Claude Opus 4.6 commit 281c0f110360f5fb4109a527306bf55ab2f5fc5c Author: Lawrence Hook Date: Sun Mar 1 16:46:31 2026 -0500 Add "Active" sidebar tab and soften selected sidebar styling Adds "all" and "active" tabs at the top of the sidebar. "Active" shows all currently-enabled options in one place; "all" explicitly resets to the default view. Also lightens the sidebar selected state from solid black to a subtle tint. Co-Authored-By: Claude Opus 4.6 commit 56b119ed98cb37cad65ed2fce7a7e34a62e3060b Author: Lawrence Hook Date: Sun Feb 22 05:01:00 2026 -0500 Cache grandfathered emails in memory at startup Co-Authored-By: Claude Opus 4.6 commit afed16a38a87332fd512c16e4cac80f04da6fa26 Author: Lawrence Hook Date: Sun Feb 22 04:47:51 2026 -0500 Add tests for cache TTL and webhook ordering race condition Co-Authored-By: Claude Opus 4.6 commit 361a8caf847a9ef48406db1857307d38494a5432 Author: Lawrence Hook Date: Sun Feb 22 04:34:00 2026 -0500 Harden license cache: 10s TTL and skip incomplete subscription.created Co-Authored-By: Claude Opus 4.6 commit d857ae8c266b5f19b5349584bcfa6215adae0fa3 Author: Lawrence Hook Date: Sat Feb 21 18:24:33 2026 -0500 Update README: replace donations blurb with premium link Co-Authored-By: Claude Opus 4.6 commit c473314ca546092b88b5f2d613cc5d5b8f4e695d Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Sat Feb 21 18:22:39 2026 -0500 Bump fast-xml-parser and @aws-sdk/xml-builder in /server (#201) Bumps [fast-xml-parser](https://github.com/NaturalIntelligence/fast-xml-parser) and [@aws-sdk/xml-builder](https://github.com/aws/aws-sdk-js-v3/tree/HEAD/packages-internal/xml-builder). These dependencies needed to be updated together. Updates `fast-xml-parser` from 5.3.4 to 5.3.6 - [Release notes](https://github.com/NaturalIntelligence/fast-xml-parser/releases) - [Changelog](https://github.com/NaturalIntelligence/fast-xml-parser/blob/master/CHANGELOG.md) - [Commits](https://github.com/NaturalIntelligence/fast-xml-parser/compare/v5.3.4...v5.3.6) Updates `@aws-sdk/xml-builder` from 3.972.4 to 3.972.5 - [Release notes](https://github.com/aws/aws-sdk-js-v3/releases) - [Changelog](https://github.com/aws/aws-sdk-js-v3/blob/main/packages-internal/xml-builder/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-js-v3/commits/HEAD/packages-internal/xml-builder) --- updated-dependencies: - dependency-name: fast-xml-parser dependency-version: 5.3.6 dependency-type: indirect - dependency-name: "@aws-sdk/xml-builder" dependency-version: 3.972.5 dependency-type: indirect ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 49e1a5d9ba7022b9f69afe399edff9cf1ad92eca Author: Lawrence Hook Date: Sat Feb 21 17:30:29 2026 -0500 Fix license cache not updating after checkout The checkout.session.completed webhook only sent a welcome email but did not update the subscription cache, so users kept getting cached free status after paying. Co-Authored-By: Claude Opus 4.6 commit c5734518c407ce436d0b281ab61c18ca21e0f6bb Author: Lawrence Hook Date: Sun Feb 15 19:10:05 2026 -0500 Remove TESTING.md and DEPLOYMENT.md Co-Authored-By: Claude Opus 4.6 commit 6ed3ad02910ea4d39dc578b8e87de092b76508ff Author: Lawrence Hook Date: Sun Feb 15 19:08:03 2026 -0500 Remove obsolete docs and update TODO for YouTube DOM change Co-Authored-By: Claude Opus 4.6 commit 83693504e3b386f681ac0cb678180eb9d19c5acd Author: Lawrence Hook Date: Sun Feb 15 18:54:29 2026 -0500 update versions commit 27a6ae6f18f2b2942a79f293d87164ae7609f0b5 Merge: 3f4b42b d6e642b Author: Lawrence Hook Date: Sun Feb 15 18:53:37 2026 -0500 Merge branch 'released' commit 3f4b42b2dea44f957449816a79fdf5f4d2399312 Author: Lawrence Hook Date: Sun Feb 15 18:49:28 2026 -0500 Update qs to 6.14.2 to fix low-severity DoS vulnerability Co-Authored-By: Claude Opus 4.6 commit 8825666512ff6bfb16b237f3180af4ed259583a8 Author: Lawrence Hook Date: Sun Feb 15 18:45:37 2026 -0500 Merge the monetization feature branch. Add RYS Premium subscription system Introduce optional paid tier with sign-in, Stripe billing, and premium feature gating. Core features remain free. Includes Node.js server for auth, payments, and license management. Extension changes: - Sign-in flow with magic link email - Premium/upgrade/account modals with branded UI - Premium feature gating (60+ advanced settings) - Password lock and scheduling (premium) - Fix "hide all but first row" for YouTube's new flat homepage DOM - Hide sidebar ad panels by default - Enable "Hide all Shorts" by default Server: - Magic link auth with rate limiting - Stripe checkout, webhooks, and billing portal - JWT-based license tokens - AWS SES transactional emails (welcome, sign-in, cancellation) - Grandfathered donor support Also adds CI workflow, server test suite, and extension unit tests. commit a76f69804b46ad8cd579e3b325683b0428cdcf3a Author: Lawrence Hook Date: Sun Feb 8 20:45:35 2026 -0500 update description in README Co-Authored-By: Claude Opus 4.6 commit b5d1384ca88c6c16d190cf8d5a189be7b6db7c53 Author: Lawrence Hook Date: Sun Feb 8 20:44:26 2026 -0500 update "Why I made it" section in README Co-Authored-By: Claude Opus 4.6 commit 18b64708b689641aabe367781f9889552717a363 Author: Lawrence Hook Date: Sun Feb 8 20:14:23 2026 -0500 update version to 4.3.71 --- server/package-lock.json | 505 ++++++++++++++++++++++++++++++++++- server/package.json | 2 + server/src/config.js | 4 +- server/src/index.js | 1 + server/src/routes/auth.js | 29 +- server/src/services/email.js | 20 ++ server/src/storage/index.js | 357 ++++++++++++------------- server/tests/auth.test.js | 14 +- server/tests/email.test.js | 77 ++++++ server/tests/setup.js | 3 +- server/tests/storage.test.js | 37 +-- src/chrome_manifest.json | 2 +- src/firefox_manifest.json | 2 +- src/options/body.css | 7 + src/options/main.html | 1 + 15 files changed, 841 insertions(+), 220 deletions(-) create mode 100644 server/tests/email.test.js diff --git a/server/package-lock.json b/server/package-lock.json index 6c6113c..dac0f3c 100644 --- a/server/package-lock.json +++ b/server/package-lock.json @@ -9,6 +9,8 @@ "version": "1.0.0", "dependencies": { "@aws-sdk/client-ses": "^3.1000.0", + "@aws-sdk/client-sesv2": "^3.1000.0", + "better-sqlite3": "^11.0.0", "cors": "^2.8.5", "dotenv": "^17.2.3", "express": "^4.18.2", @@ -200,6 +202,57 @@ "node": ">=20.0.0" } }, + "node_modules/@aws-sdk/client-sesv2": { + "version": "3.1000.0", + "resolved": "https://registry.npmjs.org/@aws-sdk/client-sesv2/-/client-sesv2-3.1000.0.tgz", + "integrity": "sha512-RImSQzSPCMhWC+uR0LEkvpY2/FQShP8i67I6Nl4ZOsggO1zLbwhw9WdNrb2b0Lcq42ptaGFyk2j767GHrt+YiA==", + "license": "Apache-2.0", + "dependencies": { + "@aws-crypto/sha256-browser": "5.2.0", + "@aws-crypto/sha256-js": "5.2.0", + "@aws-sdk/core": "^3.973.15", + "@aws-sdk/credential-provider-node": "^3.972.14", + "@aws-sdk/middleware-host-header": "^3.972.6", + "@aws-sdk/middleware-logger": "^3.972.6", + "@aws-sdk/middleware-recursion-detection": "^3.972.6", + "@aws-sdk/middleware-user-agent": "^3.972.15", + "@aws-sdk/region-config-resolver": "^3.972.6", + "@aws-sdk/signature-v4-multi-region": "^3.996.3", + "@aws-sdk/types": "^3.973.4", + "@aws-sdk/util-endpoints": "^3.996.3", + "@aws-sdk/util-user-agent-browser": "^3.972.6", + "@aws-sdk/util-user-agent-node": "^3.973.0", + "@smithy/config-resolver": "^4.4.9", + "@smithy/core": "^3.23.6", + "@smithy/fetch-http-handler": "^5.3.11", + "@smithy/hash-node": "^4.2.10", + "@smithy/invalid-dependency": "^4.2.10", + "@smithy/middleware-content-length": "^4.2.10", + "@smithy/middleware-endpoint": "^4.4.20", + "@smithy/middleware-retry": "^4.4.37", + "@smithy/middleware-serde": "^4.2.11", + "@smithy/middleware-stack": "^4.2.10", + "@smithy/node-config-provider": "^4.3.10", + "@smithy/node-http-handler": "^4.4.12", + "@smithy/protocol-http": "^5.3.10", + "@smithy/smithy-client": "^4.12.0", + "@smithy/types": "^4.13.0", + "@smithy/url-parser": "^4.2.10", + "@smithy/util-base64": "^4.3.1", + "@smithy/util-body-length-browser": "^4.2.1", + "@smithy/util-body-length-node": "^4.2.2", + "@smithy/util-defaults-mode-browser": "^4.3.36", + "@smithy/util-defaults-mode-node": "^4.2.39", + "@smithy/util-endpoints": "^3.3.1", + "@smithy/util-middleware": "^4.2.10", + "@smithy/util-retry": "^4.2.10", + "@smithy/util-utf8": "^4.2.1", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=20.0.0" + } + }, "node_modules/@aws-sdk/core": { "version": "3.973.15", "resolved": "https://registry.npmjs.org/@aws-sdk/core/-/core-3.973.15.tgz", @@ -427,6 +480,31 @@ "node": ">=20.0.0" } }, + "node_modules/@aws-sdk/middleware-sdk-s3": { + "version": "3.972.15", + "resolved": "https://registry.npmjs.org/@aws-sdk/middleware-sdk-s3/-/middleware-sdk-s3-3.972.15.tgz", + "integrity": "sha512-WDLgssevOU5BFx1s8jA7jj6cE5HuImz28sy9jKOaVtz0AW1lYqSzotzdyiybFaBcQTs5zxXOb2pUfyMxgEKY3Q==", + "license": "Apache-2.0", + "dependencies": { + "@aws-sdk/core": "^3.973.15", + "@aws-sdk/types": "^3.973.4", + "@aws-sdk/util-arn-parser": "^3.972.2", + "@smithy/core": "^3.23.6", + "@smithy/node-config-provider": "^4.3.10", + "@smithy/protocol-http": "^5.3.10", + "@smithy/signature-v4": "^5.3.10", + "@smithy/smithy-client": "^4.12.0", + "@smithy/types": "^4.13.0", + "@smithy/util-config-provider": "^4.2.1", + "@smithy/util-middleware": "^4.2.10", + "@smithy/util-stream": "^4.5.15", + "@smithy/util-utf8": "^4.2.1", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=20.0.0" + } + }, "node_modules/@aws-sdk/middleware-user-agent": { "version": "3.972.15", "resolved": "https://registry.npmjs.org/@aws-sdk/middleware-user-agent/-/middleware-user-agent-3.972.15.tgz", @@ -510,6 +588,23 @@ "node": ">=20.0.0" } }, + "node_modules/@aws-sdk/signature-v4-multi-region": { + "version": "3.996.3", + "resolved": "https://registry.npmjs.org/@aws-sdk/signature-v4-multi-region/-/signature-v4-multi-region-3.996.3.tgz", + "integrity": "sha512-gQYI/Buwp0CAGQxY7mR5VzkP56rkWq2Y1ROkFuXh5XY94DsSjJw62B3I0N0lysQmtwiL2ht2KHI9NylM/RP4FA==", + "license": "Apache-2.0", + "dependencies": { + "@aws-sdk/middleware-sdk-s3": "^3.972.15", + "@aws-sdk/types": "^3.973.4", + "@smithy/protocol-http": "^5.3.10", + "@smithy/signature-v4": "^5.3.10", + "@smithy/types": "^4.13.0", + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=20.0.0" + } + }, "node_modules/@aws-sdk/token-providers": { "version": "3.999.0", "resolved": "https://registry.npmjs.org/@aws-sdk/token-providers/-/token-providers-3.999.0.tgz", @@ -541,6 +636,18 @@ "node": ">=20.0.0" } }, + "node_modules/@aws-sdk/util-arn-parser": { + "version": "3.972.2", + "resolved": "https://registry.npmjs.org/@aws-sdk/util-arn-parser/-/util-arn-parser-3.972.2.tgz", + "integrity": "sha512-VkykWbqMjlSgBFDyrY3nOSqupMc6ivXuGmvci6Q3NnLq5kC+mKQe2QBZ4nrWRE/jqOxeFP2uYzLtwncYYcvQDg==", + "license": "Apache-2.0", + "dependencies": { + "tslib": "^2.6.2" + }, + "engines": { + "node": ">=20.0.0" + } + }, "node_modules/@aws-sdk/util-endpoints": { "version": "3.996.3", "resolved": "https://registry.npmjs.org/@aws-sdk/util-endpoints/-/util-endpoints-3.996.3.tgz", @@ -1287,6 +1394,26 @@ "dev": true, "license": "MIT" }, + "node_modules/base64-js": { + "version": "1.5.1", + "resolved": "https://registry.npmjs.org/base64-js/-/base64-js-1.5.1.tgz", + "integrity": "sha512-AKpaYlHn8t4SVbOHCy+b5+KKgvR4vrsD8vbvrbiQJps7fKDTkjkDry6ji0rUJjC0kzbNePLwzxq8iypo41qeWA==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/feross" + }, + { + "type": "patreon", + "url": "https://www.patreon.com/feross" + }, + { + "type": "consulting", + "url": "https://feross.org/support" + } + ], + "license": "MIT" + }, "node_modules/basic-auth": { "version": "2.0.1", "resolved": "https://registry.npmjs.org/basic-auth/-/basic-auth-2.0.1.tgz", @@ -1305,6 +1432,37 @@ "integrity": "sha512-Gd2UZBJDkXlY7GbJxfsE8/nvKkUEU1G38c1siN6QP6a9PT9MmHB8GnpscSmMJSoF8LOIrt8ud/wPtojys4G6+g==", "license": "MIT" }, + "node_modules/better-sqlite3": { + "version": "11.10.0", + "resolved": "https://registry.npmjs.org/better-sqlite3/-/better-sqlite3-11.10.0.tgz", + "integrity": "sha512-EwhOpyXiOEL/lKzHz9AW1msWFNzGc/z+LzeB3/jnFJpxu+th2yqvzsSWas1v9jgs9+xiXJcD5A8CJxAG2TaghQ==", + "hasInstallScript": true, + "license": "MIT", + "dependencies": { + "bindings": "^1.5.0", + "prebuild-install": "^7.1.1" + } + }, + "node_modules/bindings": { + "version": "1.5.0", + "resolved": "https://registry.npmjs.org/bindings/-/bindings-1.5.0.tgz", + "integrity": "sha512-p2q/t/mhvuOj/UeLlV6566GD/guowlr0hHxClI0W9m7MWYkL1F0hLo+0Aexs9HSPCtR1SXQ0TD3MMKrXZajbiQ==", + "license": "MIT", + "dependencies": { + "file-uri-to-path": "1.0.0" + } + }, + "node_modules/bl": { + "version": "4.1.0", + "resolved": "https://registry.npmjs.org/bl/-/bl-4.1.0.tgz", + "integrity": "sha512-1W07cM9gS6DcLperZfFSj+bWLtaPGSOHWhPiGzXmvVJbRLdG82sH/Kn8EtW1VqWVA54AKf2h5k5BbnIbwF3h6w==", + "license": "MIT", + "dependencies": { + "buffer": "^5.5.0", + "inherits": "^2.0.4", + "readable-stream": "^3.4.0" + } + }, "node_modules/body-parser": { "version": "1.20.4", "resolved": "https://registry.npmjs.org/body-parser/-/body-parser-1.20.4.tgz", @@ -1335,6 +1493,30 @@ "integrity": "sha512-tzPjzCxygAKWFOJP011oxFHs57HzIhOEracIgAePE4pqB3LikALKnSzUyU4MGs9/iCEUuHlAJTjTc5M+u7YEGg==", "license": "MIT" }, + "node_modules/buffer": { + "version": "5.7.1", + "resolved": "https://registry.npmjs.org/buffer/-/buffer-5.7.1.tgz", + "integrity": "sha512-EHcyIPBQ4BSGlvjB16k5KgAJ27CIsHY/2JBmCRReo48y9rQ3MaUzWX3KVlBa4U7MyX02HdVj0K7C3WaB3ju7FQ==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/feross" + }, + { + "type": "patreon", + "url": "https://www.patreon.com/feross" + }, + { + "type": "consulting", + "url": "https://feross.org/support" + } + ], + "license": "MIT", + "dependencies": { + "base64-js": "^1.3.1", + "ieee754": "^1.1.13" + } + }, "node_modules/buffer-equal-constant-time": { "version": "1.0.1", "resolved": "https://registry.npmjs.org/buffer-equal-constant-time/-/buffer-equal-constant-time-1.0.1.tgz", @@ -1379,6 +1561,12 @@ "url": "https://github.com/sponsors/ljharb" } }, + "node_modules/chownr": { + "version": "1.1.4", + "resolved": "https://registry.npmjs.org/chownr/-/chownr-1.1.4.tgz", + "integrity": "sha512-jJ0bqzaylmJtVnNgzTeSOs8DPavpbYgEr/b0YL8/2GO3xJEhInFmhKMUnEJQjZumK7KXGFhUy89PrsJWlakBVg==", + "license": "ISC" + }, "node_modules/combined-stream": { "version": "1.0.8", "resolved": "https://registry.npmjs.org/combined-stream/-/combined-stream-1.0.8.tgz", @@ -1467,6 +1655,30 @@ "ms": "2.0.0" } }, + "node_modules/decompress-response": { + "version": "6.0.0", + "resolved": "https://registry.npmjs.org/decompress-response/-/decompress-response-6.0.0.tgz", + "integrity": "sha512-aW35yZM6Bb/4oJlZncMH2LCoZtJXTRxES17vE3hoRiowU2kWHaJKFkSBDnDR+cm9J+9QhXmREyIfv0pji9ejCQ==", + "license": "MIT", + "dependencies": { + "mimic-response": "^3.1.0" + }, + "engines": { + "node": ">=10" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/deep-extend": { + "version": "0.6.0", + "resolved": "https://registry.npmjs.org/deep-extend/-/deep-extend-0.6.0.tgz", + "integrity": "sha512-LOHxIOaPYdHlJRtCQfDIVZtfw/ufM8+rVj649RIHzcm/vGwQRXFt6OPqIFWsm2XEMrNIEtWR64sY1LEKD2vAOA==", + "license": "MIT", + "engines": { + "node": ">=4.0.0" + } + }, "node_modules/delayed-stream": { "version": "1.0.0", "resolved": "https://registry.npmjs.org/delayed-stream/-/delayed-stream-1.0.0.tgz", @@ -1496,6 +1708,15 @@ "npm": "1.2.8000 || >= 1.4.16" } }, + "node_modules/detect-libc": { + "version": "2.1.2", + "resolved": "https://registry.npmjs.org/detect-libc/-/detect-libc-2.1.2.tgz", + "integrity": "sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ==", + "license": "Apache-2.0", + "engines": { + "node": ">=8" + } + }, "node_modules/dezalgo": { "version": "1.0.4", "resolved": "https://registry.npmjs.org/dezalgo/-/dezalgo-1.0.4.tgz", @@ -1557,6 +1778,15 @@ "node": ">= 0.8" } }, + "node_modules/end-of-stream": { + "version": "1.4.5", + "resolved": "https://registry.npmjs.org/end-of-stream/-/end-of-stream-1.4.5.tgz", + "integrity": "sha512-ooEGc6HP26xXq/N+GCGOT0JKCLDGrq2bQUZrQ7gyrJiZANJ/8YDTxTpQBXGMn+WbIQXNVpyWymm7KYVICQnyOg==", + "license": "MIT", + "dependencies": { + "once": "^1.4.0" + } + }, "node_modules/es-define-property": { "version": "1.0.1", "resolved": "https://registry.npmjs.org/es-define-property/-/es-define-property-1.0.1.tgz", @@ -1618,6 +1848,15 @@ "node": ">= 0.6" } }, + "node_modules/expand-template": { + "version": "2.0.3", + "resolved": "https://registry.npmjs.org/expand-template/-/expand-template-2.0.3.tgz", + "integrity": "sha512-XYfuKMvj4O35f/pOXLObndIRvyQ+/+6AhODh+OKWj9S9498pHHn/IMszH+gt0fBCRWMNfk1ZSp5x3AifmnI2vg==", + "license": "(MIT OR WTFPL)", + "engines": { + "node": ">=6" + } + }, "node_modules/express": { "version": "4.22.1", "resolved": "https://registry.npmjs.org/express/-/express-4.22.1.tgz", @@ -1702,6 +1941,12 @@ "fxparser": "src/cli/cli.js" } }, + "node_modules/file-uri-to-path": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/file-uri-to-path/-/file-uri-to-path-1.0.0.tgz", + "integrity": "sha512-0Zt+s3L7Vf1biwWZ29aARiVYLx7iMGnEUl9x33fbB/j3jR81u/O2LbqK+Bm1CDSNDKVtJ/YjwY7TUd5SkeLQLw==", + "license": "MIT" + }, "node_modules/finalhandler": { "version": "1.3.2", "resolved": "https://registry.npmjs.org/finalhandler/-/finalhandler-1.3.2.tgz", @@ -1771,6 +2016,12 @@ "node": ">= 0.6" } }, + "node_modules/fs-constants": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/fs-constants/-/fs-constants-1.0.0.tgz", + "integrity": "sha512-y6OAwoSIf7FyjMIv94u+b5rdheZEjzR63GTyZJm5qh4Bi+2YgwLCcI/fPFZkL5PSixOt6ZNKm+w+Hfp/Bciwow==", + "license": "MIT" + }, "node_modules/function-bind": { "version": "1.1.2", "resolved": "https://registry.npmjs.org/function-bind/-/function-bind-1.1.2.tgz", @@ -1817,6 +2068,12 @@ "node": ">= 0.4" } }, + "node_modules/github-from-package": { + "version": "0.0.0", + "resolved": "https://registry.npmjs.org/github-from-package/-/github-from-package-0.0.0.tgz", + "integrity": "sha512-SyHy3T1v2NUXn29OsWdxmK6RwHD+vkj3v8en8AOBZ1wBQ/hCAQ5bAQTD02kW4W9tUp/3Qh6J8r9EvntiyCmOOw==", + "license": "MIT" + }, "node_modules/gopd": { "version": "1.2.0", "resolved": "https://registry.npmjs.org/gopd/-/gopd-1.2.0.tgz", @@ -1901,12 +2158,38 @@ "node": ">=0.10.0" } }, + "node_modules/ieee754": { + "version": "1.2.1", + "resolved": "https://registry.npmjs.org/ieee754/-/ieee754-1.2.1.tgz", + "integrity": "sha512-dcyqhDvX1C46lXZcVqCpK+FtMRQVdIMN6/Df5js2zouUsqG7I6sFxitIC+7KYK29KdXOLHdu9zL4sFnoVQnqaA==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/feross" + }, + { + "type": "patreon", + "url": "https://www.patreon.com/feross" + }, + { + "type": "consulting", + "url": "https://feross.org/support" + } + ], + "license": "BSD-3-Clause" + }, "node_modules/inherits": { "version": "2.0.4", "resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.4.tgz", "integrity": "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==", "license": "ISC" }, + "node_modules/ini": { + "version": "1.3.8", + "resolved": "https://registry.npmjs.org/ini/-/ini-1.3.8.tgz", + "integrity": "sha512-JV/yugV2uzW5iMRSiZAyDtQd+nxtUnjeLt0acNdw98kKLrvuRVyB80tsREOE7yvGVgalhZ6RNXCmEHkUKBKxew==", + "license": "ISC" + }, "node_modules/ipaddr.js": { "version": "1.9.1", "resolved": "https://registry.npmjs.org/ipaddr.js/-/ipaddr.js-1.9.1.tgz", @@ -2076,6 +2359,33 @@ "node": ">= 0.6" } }, + "node_modules/mimic-response": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/mimic-response/-/mimic-response-3.1.0.tgz", + "integrity": "sha512-z0yWI+4FDrrweS8Zmt4Ej5HdJmky15+L2e6Wgn3+iK5fWzb6T3fhNFq2+MeTRb064c6Wr4N/wv0DzQTjNzHNGQ==", + "license": "MIT", + "engines": { + "node": ">=10" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/minimist": { + "version": "1.2.8", + "resolved": "https://registry.npmjs.org/minimist/-/minimist-1.2.8.tgz", + "integrity": "sha512-2yyAR8qBkN3YuheJanUpWC5U3bb5osDywNB8RzDVlDwDHbocAJveqqj1u8+SVD7jkWT4yvsHCpWqqWqAxb0zCA==", + "license": "MIT", + "funding": { + "url": "https://github.com/sponsors/ljharb" + } + }, + "node_modules/mkdirp-classic": { + "version": "0.5.3", + "resolved": "https://registry.npmjs.org/mkdirp-classic/-/mkdirp-classic-0.5.3.tgz", + "integrity": "sha512-gKLcREMhtuZRwRAfqP3RFW+TK4JqApVBtOIftVgjuABpAtpxhPGaDcfvbhNvD0B8iD1oUr/txX35NjcaY6Ns/A==", + "license": "MIT" + }, "node_modules/morgan": { "version": "1.10.1", "resolved": "https://registry.npmjs.org/morgan/-/morgan-1.10.1.tgz", @@ -2110,6 +2420,12 @@ "integrity": "sha512-Tpp60P6IUJDTuOq/5Z8cdskzJujfwqfOTkrwIwj7IRISpnkJnT6SyJ4PCPnGMoFjC9ddhal5KVIYtAt97ix05A==", "license": "MIT" }, + "node_modules/napi-build-utils": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/napi-build-utils/-/napi-build-utils-2.0.0.tgz", + "integrity": "sha512-GEbrYkbfF7MoNaoh2iGG84Mnf/WZfB0GdGEsM8wz7Expx/LlWf5U8t9nvJKXSp3qr5IsEbK04cBGhol/KwOsWA==", + "license": "MIT" + }, "node_modules/negotiator": { "version": "0.6.3", "resolved": "https://registry.npmjs.org/negotiator/-/negotiator-0.6.3.tgz", @@ -2119,6 +2435,18 @@ "node": ">= 0.6" } }, + "node_modules/node-abi": { + "version": "3.87.0", + "resolved": "https://registry.npmjs.org/node-abi/-/node-abi-3.87.0.tgz", + "integrity": "sha512-+CGM1L1CgmtheLcBuleyYOn7NWPVu0s0EJH2C4puxgEZb9h8QpR9G2dBfZJOAUhi7VQxuBPMd0hiISWcTyiYyQ==", + "license": "MIT", + "dependencies": { + "semver": "^7.3.5" + }, + "engines": { + "node": ">=10" + } + }, "node_modules/object-assign": { "version": "4.1.1", "resolved": "https://registry.npmjs.org/object-assign/-/object-assign-4.1.1.tgz", @@ -2165,7 +2493,6 @@ "version": "1.4.0", "resolved": "https://registry.npmjs.org/once/-/once-1.4.0.tgz", "integrity": "sha512-lNaJgI+2Q5URQBkccEKHTQOPaXdUxnZZElQTZY0MFUAuaEqe1E+Nyvgdz/aIyNi6Z9MzO5dv1H8n58/GELp3+w==", - "dev": true, "license": "ISC", "dependencies": { "wrappy": "1" @@ -2186,6 +2513,33 @@ "integrity": "sha512-RA1GjUVMnvYFxuqovrEqZoxxW5NUZqbwKtYz/Tt7nXerk0LbLblQmrsgdeOxV5SFHf0UDggjS/bSeOZwt1pmEQ==", "license": "MIT" }, + "node_modules/prebuild-install": { + "version": "7.1.3", + "resolved": "https://registry.npmjs.org/prebuild-install/-/prebuild-install-7.1.3.tgz", + "integrity": "sha512-8Mf2cbV7x1cXPUILADGI3wuhfqWvtiLA1iclTDbFRZkgRQS0NqsPZphna9V+HyTEadheuPmjaJMsbzKQFOzLug==", + "deprecated": "No longer maintained. Please contact the author of the relevant native addon; alternatives are available.", + "license": "MIT", + "dependencies": { + "detect-libc": "^2.0.0", + "expand-template": "^2.0.3", + "github-from-package": "0.0.0", + "minimist": "^1.2.3", + "mkdirp-classic": "^0.5.3", + "napi-build-utils": "^2.0.0", + "node-abi": "^3.3.0", + "pump": "^3.0.0", + "rc": "^1.2.7", + "simple-get": "^4.0.0", + "tar-fs": "^2.0.0", + "tunnel-agent": "^0.6.0" + }, + "bin": { + "prebuild-install": "bin.js" + }, + "engines": { + "node": ">=10" + } + }, "node_modules/proxy-addr": { "version": "2.0.7", "resolved": "https://registry.npmjs.org/proxy-addr/-/proxy-addr-2.0.7.tgz", @@ -2199,6 +2553,16 @@ "node": ">= 0.10" } }, + "node_modules/pump": { + "version": "3.0.4", + "resolved": "https://registry.npmjs.org/pump/-/pump-3.0.4.tgz", + "integrity": "sha512-VS7sjc6KR7e1ukRFhQSY5LM2uBWAUPiOPa/A3mkKmiMwSmRFUITt0xuj+/lesgnCv+dPIEYlkzrcyXgquIHMcA==", + "license": "MIT", + "dependencies": { + "end-of-stream": "^1.1.0", + "once": "^1.3.1" + } + }, "node_modules/qs": { "version": "6.14.2", "resolved": "https://registry.npmjs.org/qs/-/qs-6.14.2.tgz", @@ -2238,6 +2602,35 @@ "node": ">= 0.8" } }, + "node_modules/rc": { + "version": "1.2.8", + "resolved": "https://registry.npmjs.org/rc/-/rc-1.2.8.tgz", + "integrity": "sha512-y3bGgqKj3QBdxLbLkomlohkvsA8gdAiUQlSBJnBhfn+BPxg4bc62d8TcBW15wavDfgexCgccckhcZvywyQYPOw==", + "license": "(BSD-2-Clause OR MIT OR Apache-2.0)", + "dependencies": { + "deep-extend": "^0.6.0", + "ini": "~1.3.0", + "minimist": "^1.2.0", + "strip-json-comments": "~2.0.1" + }, + "bin": { + "rc": "cli.js" + } + }, + "node_modules/readable-stream": { + "version": "3.6.2", + "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-3.6.2.tgz", + "integrity": "sha512-9u/sniCrY3D5WdsERHzHE4G2YCXqoG5FTHUiCC4SIbr6XcLZBY05ya9EKjYek9O5xOAwjGq+1JdGBAS7Q9ScoA==", + "license": "MIT", + "dependencies": { + "inherits": "^2.0.3", + "string_decoder": "^1.1.1", + "util-deprecate": "^1.0.1" + }, + "engines": { + "node": ">= 6" + } + }, "node_modules/safe-buffer": { "version": "5.2.1", "resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.2.1.tgz", @@ -2399,6 +2792,51 @@ "url": "https://github.com/sponsors/ljharb" } }, + "node_modules/simple-concat": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/simple-concat/-/simple-concat-1.0.1.tgz", + "integrity": "sha512-cSFtAPtRhljv69IK0hTVZQ+OfE9nePi/rtJmw5UjHeVyVroEqJXP1sFztKUy1qU+xvz3u/sfYJLa947b7nAN2Q==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/feross" + }, + { + "type": "patreon", + "url": "https://www.patreon.com/feross" + }, + { + "type": "consulting", + "url": "https://feross.org/support" + } + ], + "license": "MIT" + }, + "node_modules/simple-get": { + "version": "4.0.1", + "resolved": "https://registry.npmjs.org/simple-get/-/simple-get-4.0.1.tgz", + "integrity": "sha512-brv7p5WgH0jmQJr1ZDDfKDOSeWWg+OVypG99A/5vYGPqJ6pxiaHLy8nxtFjBA7oMa01ebA9gfh1uMCFqOuXxvA==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/feross" + }, + { + "type": "patreon", + "url": "https://www.patreon.com/feross" + }, + { + "type": "consulting", + "url": "https://feross.org/support" + } + ], + "license": "MIT", + "dependencies": { + "decompress-response": "^6.0.0", + "once": "^1.3.1", + "simple-concat": "^1.0.0" + } + }, "node_modules/statuses": { "version": "2.0.2", "resolved": "https://registry.npmjs.org/statuses/-/statuses-2.0.2.tgz", @@ -2408,6 +2846,24 @@ "node": ">= 0.8" } }, + "node_modules/string_decoder": { + "version": "1.3.0", + "resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-1.3.0.tgz", + "integrity": "sha512-hkRX8U1WjJFd8LsDJ2yQ/wWWxaopEsABU1XfkM8A+j0+85JAGppt16cr1Whg6KIbb4okU6Mql6BOj+uup/wKeA==", + "license": "MIT", + "dependencies": { + "safe-buffer": "~5.2.0" + } + }, + "node_modules/strip-json-comments": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/strip-json-comments/-/strip-json-comments-2.0.1.tgz", + "integrity": "sha512-4gB8na07fecVVkOI6Rs4e7T6NOTki5EmL7TUduTs6bu3EdnSycntVJ4re8kgZA+wx9IueI2Y11bfbgwtzuE0KQ==", + "license": "MIT", + "engines": { + "node": ">=0.10.0" + } + }, "node_modules/stripe": { "version": "14.25.0", "resolved": "https://registry.npmjs.org/stripe/-/stripe-14.25.0.tgz", @@ -2509,6 +2965,34 @@ "node": ">=6.4.0" } }, + "node_modules/tar-fs": { + "version": "2.1.4", + "resolved": "https://registry.npmjs.org/tar-fs/-/tar-fs-2.1.4.tgz", + "integrity": "sha512-mDAjwmZdh7LTT6pNleZ05Yt65HC3E+NiQzl672vQG38jIrehtJk/J3mNwIg+vShQPcLF/LV7CMnDW6vjj6sfYQ==", + "license": "MIT", + "dependencies": { + "chownr": "^1.1.1", + "mkdirp-classic": "^0.5.2", + "pump": "^3.0.0", + "tar-stream": "^2.1.4" + } + }, + "node_modules/tar-stream": { + "version": "2.2.0", + "resolved": "https://registry.npmjs.org/tar-stream/-/tar-stream-2.2.0.tgz", + "integrity": "sha512-ujeqbceABgwMZxEJnk2HDY2DlnUZ+9oEcb1KzTVfYHio0UE6dG71n60d8D2I4qNvleWrrXpmjpt7vZeF1LnMZQ==", + "license": "MIT", + "dependencies": { + "bl": "^4.0.3", + "end-of-stream": "^1.4.1", + "fs-constants": "^1.0.0", + "inherits": "^2.0.3", + "readable-stream": "^3.1.1" + }, + "engines": { + "node": ">=6" + } + }, "node_modules/toidentifier": { "version": "1.0.1", "resolved": "https://registry.npmjs.org/toidentifier/-/toidentifier-1.0.1.tgz", @@ -2524,6 +3008,18 @@ "integrity": "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w==", "license": "0BSD" }, + "node_modules/tunnel-agent": { + "version": "0.6.0", + "resolved": "https://registry.npmjs.org/tunnel-agent/-/tunnel-agent-0.6.0.tgz", + "integrity": "sha512-McnNiV1l8RYeY8tBgEpuodCC1mLUdbSN+CYBL7kJsJNInOP8UjDDEwdk6Mw60vdLLrr5NHKZhMAOSrR2NZuQ+w==", + "license": "Apache-2.0", + "dependencies": { + "safe-buffer": "^5.0.1" + }, + "engines": { + "node": "*" + } + }, "node_modules/type-is": { "version": "1.6.18", "resolved": "https://registry.npmjs.org/type-is/-/type-is-1.6.18.tgz", @@ -2552,6 +3048,12 @@ "node": ">= 0.8" } }, + "node_modules/util-deprecate": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/util-deprecate/-/util-deprecate-1.0.2.tgz", + "integrity": "sha512-EPD5q1uXyFxJpCrLnCc1nHnq3gOa6DZBocAIiI2TaSCA7VCJ1UJDMagCzIkXNsUYfD1daK//LTEQ8xiIbrHtcw==", + "license": "MIT" + }, "node_modules/utils-merge": { "version": "1.0.1", "resolved": "https://registry.npmjs.org/utils-merge/-/utils-merge-1.0.1.tgz", @@ -2587,7 +3089,6 @@ "version": "1.0.2", "resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz", "integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==", - "dev": true, "license": "ISC" } } diff --git a/server/package.json b/server/package.json index d8bc66a..b21f956 100644 --- a/server/package.json +++ b/server/package.json @@ -10,6 +10,8 @@ }, "dependencies": { "@aws-sdk/client-ses": "^3.1000.0", + "@aws-sdk/client-sesv2": "^3.1000.0", + "better-sqlite3": "^11.0.0", "cors": "^2.8.5", "dotenv": "^17.2.3", "express": "^4.18.2", diff --git a/server/src/config.js b/server/src/config.js index a6d965d..4088ec8 100644 --- a/server/src/config.js +++ b/server/src/config.js @@ -18,6 +18,8 @@ module.exports = { REQUEST_ID_EXPIRY_MS: 20 * 60 * 1000, // 20 minutes RATE_LIMIT_WINDOW_MS: 60 * 60 * 1000, // 1 hour RATE_LIMIT_MAX_REQUESTS: 5, // per email per window + IP_RATE_LIMIT_WINDOW_MS: 60 * 60 * 1000, // 1 hour + IP_RATE_LIMIT_MAX_REQUESTS: 5, // per IP per window // Stripe get STRIPE_SECRET_KEY() { return process.env.STRIPE_SECRET_KEY; }, @@ -31,7 +33,5 @@ module.exports = { // File paths get DATA_DIR() { return process.env.DATA_DIR || './data'; }, - AUTH_REQUESTS_DIR: 'auth-requests', - RATE_LIMITS_DIR: 'rate-limits', GRANDFATHERED_FILE: 'grandfathered.txt', }; diff --git a/server/src/index.js b/server/src/index.js index 4ca4571..a71e536 100644 --- a/server/src/index.js +++ b/server/src/index.js @@ -98,6 +98,7 @@ function runCleanup() { try { storage.pruneExpiredAuthRequests(); storage.pruneExpiredRateLimits(); + storage.pruneExpiredIpRateLimits(); } catch (err) { console.error('Cleanup error:', err); } diff --git a/server/src/routes/auth.js b/server/src/routes/auth.js index 091c5dc..18e55f3 100644 --- a/server/src/routes/auth.js +++ b/server/src/routes/auth.js @@ -2,7 +2,7 @@ const express = require('express'); const { v4: uuidv4 } = require('uuid'); const config = require('../config'); const storage = require('../storage'); -const { sendMagicLinkEmail } = require('../services/email'); +const emailService = require('../services/email'); const { generateSessionToken } = require('../services/jwt'); const { checkPremiumStatus } = require('../services/stripe'); const { renderPage } = require('../templates'); @@ -57,7 +57,19 @@ router.post('/send-magic-link', async (req, res) => { return res.status(400).json({ error: 'Invalid email format' }); } - // Check rate limit + // Check IP rate limit + const ip = req.ip || req.connection.remoteAddress; + const ipLimit = storage.checkIpRateLimit(ip); + if (!ipLimit.allowed) { + const retryAfter = Math.ceil((ipLimit.resetTime - Date.now()) / 1000); + res.set('Retry-After', retryAfter); + return res.status(429).json({ + error: 'Too many requests. Please try again later.', + retryAfter, + }); + } + + // Check per-email rate limit const rateLimit = storage.checkRateLimit(email); if (!rateLimit.allowed) { const retryAfter = Math.ceil((rateLimit.resetTime - Date.now()) / 1000); @@ -68,15 +80,21 @@ router.post('/send-magic-link', async (req, res) => { }); } + // Validate email with SES insights + const validation = await emailService.validateEmail(email); + if (!validation.valid) { + return res.status(400).json({ error: 'This email address appears to be invalid.' }); + } + // Generate request ID and create auth request const requestId = uuidv4(); - storage.createAuthRequest(requestId, email); + storage.createAuthRequest(requestId, email, { ip }); // Send magic link email const magicLinkUrl = `${config.BASE_URL}/auth/verify?token=${requestId}`; try { const status = await checkPremiumStatus(email); - await sendMagicLinkEmail(email, magicLinkUrl, { isPremium: status.premium }); + await emailService.sendMagicLinkEmail(email, magicLinkUrl, { isPremium: status.premium }); } catch (err) { storage.deleteAuthRequest(requestId); throw err; @@ -118,6 +136,9 @@ router.get('/verify', async (req, res) => { session_token: sessionToken, }); storage.decrementRateLimit(authRequest.email); + if (authRequest.ip) { + storage.decrementIpRateLimit(authRequest.ip); + } console.log(`[auth] Email verified: ${authRequest.email}`); } diff --git a/server/src/services/email.js b/server/src/services/email.js index 7ebd49f..04e127a 100644 --- a/server/src/services/email.js +++ b/server/src/services/email.js @@ -1,7 +1,9 @@ const { SESClient, SendEmailCommand } = require('@aws-sdk/client-ses'); +const { SESv2Client, GetEmailAddressInsightsCommand } = require('@aws-sdk/client-sesv2'); const config = require('../config'); const sesClient = new SESClient({ region: config.AWS_REGION }); +const sesV2Client = new SESv2Client({ region: config.AWS_REGION }); async function sendMagicLinkEmail(email, magicLinkUrl, { isPremium = false } = {}) { const premiumBadge = isPremium @@ -188,8 +190,26 @@ async function sendCancellationEmail(email) { return sesClient.send(command); } +async function validateEmail(email) { + try { + const command = new GetEmailAddressInsightsCommand({ EmailAddress: email }); + const result = await sesV2Client.send(command, { + abortSignal: AbortSignal.timeout(2000), + }); + const verdict = result.MailboxValidation?.IsValid?.ConfidenceVerdict; + if (verdict === 'LOW') { + return { valid: false, reason: `Low confidence: ${email}` }; + } + return { valid: true }; + } catch (err) { + console.warn('[email] validateEmail error, failing open:', err.message); + return { valid: true }; + } +} + module.exports = { sendMagicLinkEmail, sendWelcomeEmail, sendCancellationEmail, + validateEmail, }; diff --git a/server/src/storage/index.js b/server/src/storage/index.js index c9bf4dc..a1db11b 100644 --- a/server/src/storage/index.js +++ b/server/src/storage/index.js @@ -1,32 +1,67 @@ const fs = require('fs'); const path = require('path'); const crypto = require('crypto'); +const Database = require('better-sqlite3'); const config = require('../config'); -// Get paths dynamically to support test environment changes -function getDataDir() { - return config.DATA_DIR; -} +// --- Database setup --- -function getAuthRequestsDir() { - return path.join(getDataDir(), config.AUTH_REQUESTS_DIR); -} +let db = null; -function getRateLimitsDir() { - return path.join(getDataDir(), config.RATE_LIMITS_DIR); +function getDb() { + if (db) return db; + + const dataDir = config.DATA_DIR; + if (!fs.existsSync(dataDir)) { + fs.mkdirSync(dataDir, { recursive: true }); + } + + db = new Database(path.join(dataDir, 'storage.db')); + db.pragma('journal_mode = WAL'); + + db.exec(` + CREATE TABLE IF NOT EXISTS auth_requests ( + request_id TEXT PRIMARY KEY, + email TEXT NOT NULL, + status TEXT NOT NULL DEFAULT 'pending', + created_at INTEGER NOT NULL, + session_token TEXT, + ip TEXT + ); + + CREATE TABLE IF NOT EXISTS email_rate_limits ( + key_hash TEXT PRIMARY KEY, + count INTEGER NOT NULL DEFAULT 0, + window_start INTEGER NOT NULL + ); + + CREATE TABLE IF NOT EXISTS ip_rate_limits ( + key_hash TEXT PRIMARY KEY, + count INTEGER NOT NULL DEFAULT 0, + window_start INTEGER NOT NULL + ); + + CREATE TABLE IF NOT EXISTS subscription_cache ( + email TEXT PRIMARY KEY, + premium INTEGER NOT NULL, + customer_id TEXT, + updated_at INTEGER NOT NULL + ); + `); + + return db; } function ensureDirectories() { - const dirs = [getDataDir(), getAuthRequestsDir(), getRateLimitsDir()]; - dirs.forEach(dir => { - if (!fs.existsSync(dir)) { - fs.mkdirSync(dir, { recursive: true }); - } - }); + getDb(); } -// Initialize on load -ensureDirectories(); +function closeDatabase() { + if (db) { + db.close(); + db = null; + } +} // --- Grandfathered Emails (loaded once at startup) --- @@ -34,7 +69,7 @@ let grandfatheredSet = null; function readGrandfatheredEmails() { try { - const filePath = path.join(getDataDir(), config.GRANDFATHERED_FILE); + const filePath = path.join(config.DATA_DIR, config.GRANDFATHERED_FILE); if (fs.existsSync(filePath)) { const lines = fs.readFileSync(filePath, 'utf8').split('\n'); return new Set(lines.map(l => l.trim().toLowerCase()).filter(l => l && !l.startsWith('#'))); @@ -55,58 +90,28 @@ function isGrandfathered(email) { return grandfatheredSet.has(email.toLowerCase()); } -// --- Subscription Cache (email -> premium status, file-backed, no in-memory cache) --- - -function getSubscriptionCachePath() { - return path.join(getDataDir(), 'subscriptions.json'); -} - -function readSubscriptionFile() { - try { - const filePath = getSubscriptionCachePath(); - if (fs.existsSync(filePath)) { - return JSON.parse(fs.readFileSync(filePath, 'utf8')); - } - } catch (err) { - console.error('Failed to read subscription cache:', err.message); - } - return {}; -} +// --- Subscription Cache --- function setSubscriptionStatus(email, premium, customerId) { email = email.toLowerCase(); - const data = readSubscriptionFile(); - data[email] = { premium, customerId, updatedAt: Date.now() }; - try { - fs.writeFileSync(getSubscriptionCachePath(), JSON.stringify(data, null, 2)); - } catch (err) { - console.error('Failed to save subscription cache:', err.message); - } + const stmt = getDb().prepare(` + INSERT INTO subscription_cache (email, premium, customer_id, updated_at) + VALUES (?, ?, ?, ?) + ON CONFLICT(email) DO UPDATE SET premium=excluded.premium, customer_id=excluded.customer_id, updated_at=excluded.updated_at + `); + stmt.run(email, premium ? 1 : 0, customerId, Date.now()); } function getSubscriptionStatus(email) { - const data = readSubscriptionFile(); - const entry = data[email.toLowerCase()]; - if (!entry) return null; - if (Date.now() - entry.updatedAt > 10000) return null; - return entry; + const row = getDb().prepare('SELECT * FROM subscription_cache WHERE email = ?').get(email.toLowerCase()); + if (!row) return null; + if (Date.now() - row.updated_at > 10000) return null; + return { premium: row.premium === 1, customerId: row.customer_id, updatedAt: row.updated_at }; } +// --- Auth Requests --- -// --- Auth Requests (file per request) --- - -function getAuthRequestFilename(requestId, timestamp) { - return `${timestamp}-${requestId}.json`; -} - -function parseAuthRequestFilename(filename) { - const match = filename.match(/^(\d+)-(.+)\.json$/); - if (!match) return null; - return { timestamp: parseInt(match[1], 10), requestId: match[2] }; -} - -function createAuthRequest(requestId, email) { - ensureDirectories(); // Ensure directory exists before write +function createAuthRequest(requestId, email, { ip } = {}) { const timestamp = Date.now(); const data = { request_id: requestId, @@ -115,171 +120,148 @@ function createAuthRequest(requestId, email) { created_at: timestamp, session_token: null, }; - const filename = getAuthRequestFilename(requestId, timestamp); - const filePath = path.join(getAuthRequestsDir(), filename); - fs.writeFileSync(filePath, JSON.stringify(data, null, 2)); + if (ip) data.ip = ip; + + const stmt = getDb().prepare(` + INSERT INTO auth_requests (request_id, email, status, created_at, session_token, ip) + VALUES (?, ?, ?, ?, ?, ?) + `); + stmt.run(requestId, email, 'pending', timestamp, null, ip || null); return data; } -function findAuthRequestFile(requestId) { - const dir = getAuthRequestsDir(); - const files = fs.readdirSync(dir); - for (const file of files) { - const parsed = parseAuthRequestFilename(file); - if (parsed && parsed.requestId === requestId) { - return path.join(dir, file); - } - } - return null; -} - function getAuthRequest(requestId) { - const filePath = findAuthRequestFile(requestId); - if (!filePath) return null; + const row = getDb().prepare('SELECT * FROM auth_requests WHERE request_id = ?').get(requestId); + if (!row) return null; - try { - const data = JSON.parse(fs.readFileSync(filePath, 'utf8')); - - // Check if expired - if (Date.now() - data.created_at > config.REQUEST_ID_EXPIRY_MS) { - // Clean up expired request - fs.unlinkSync(filePath); - return null; - } - - return data; - } catch (err) { + if (Date.now() - row.created_at > config.REQUEST_ID_EXPIRY_MS) { + getDb().prepare('DELETE FROM auth_requests WHERE request_id = ?').run(requestId); return null; } + + return { + request_id: row.request_id, + email: row.email, + status: row.status, + created_at: row.created_at, + session_token: row.session_token, + ...(row.ip ? { ip: row.ip } : {}), + }; } function updateAuthRequest(requestId, updates) { - const filePath = findAuthRequestFile(requestId); - if (!filePath) return null; + const row = getDb().prepare('SELECT * FROM auth_requests WHERE request_id = ?').get(requestId); + if (!row) return null; - try { - const data = JSON.parse(fs.readFileSync(filePath, 'utf8')); - const updated = { ...data, ...updates }; - fs.writeFileSync(filePath, JSON.stringify(updated, null, 2)); - return updated; - } catch (err) { - return null; - } + const merged = { ...row, ...updates }; + getDb().prepare(` + UPDATE auth_requests SET email=?, status=?, created_at=?, session_token=?, ip=? + WHERE request_id=? + `).run(merged.email, merged.status, merged.created_at, merged.session_token, merged.ip, requestId); + + return { + request_id: merged.request_id, + email: merged.email, + status: merged.status, + created_at: merged.created_at, + session_token: merged.session_token, + ...(merged.ip ? { ip: merged.ip } : {}), + }; } function deleteAuthRequest(requestId) { - const filePath = findAuthRequestFile(requestId); - if (filePath && fs.existsSync(filePath)) { - fs.unlinkSync(filePath); - return true; - } - return false; + const result = getDb().prepare('DELETE FROM auth_requests WHERE request_id = ?').run(requestId); + return result.changes > 0; } function pruneExpiredAuthRequests() { - const now = Date.now(); - const dir = getAuthRequestsDir(); - const files = fs.readdirSync(dir); - let pruned = 0; - - for (const file of files) { - const parsed = parseAuthRequestFilename(file); - if (parsed && now - parsed.timestamp > config.REQUEST_ID_EXPIRY_MS) { - fs.unlinkSync(path.join(dir, file)); - pruned++; - } + const cutoff = Date.now() - config.REQUEST_ID_EXPIRY_MS; + const result = getDb().prepare('DELETE FROM auth_requests WHERE created_at < ?').run(cutoff); + if (result.changes > 0) { + console.log(`Pruned ${result.changes} expired auth requests`); } - - if (pruned > 0) { - console.log(`Pruned ${pruned} expired auth requests`); - } - return pruned; + return result.changes; } -// --- Rate Limiting (file per email hash) --- +// --- Rate Limiting --- function getEmailHash(email) { return crypto.createHash('sha256').update(email.toLowerCase()).digest('hex').slice(0, 16); } -function getRateLimitFilePath(email) { - return path.join(getRateLimitsDir(), `${getEmailHash(email)}.json`); +function getIpHash(ip) { + return crypto.createHash('sha256').update(ip).digest('hex').slice(0, 16); } -function checkRateLimit(email) { - ensureDirectories(); // Ensure directory exists before write - const filePath = getRateLimitFilePath(email); +function _checkRateLimit(table, keyHash, windowMs, maxRequests) { const now = Date.now(); + const d = getDb(); - let data = { count: 0, windowStart: now }; + let row = d.prepare(`SELECT * FROM ${table} WHERE key_hash = ?`).get(keyHash); - try { - if (fs.existsSync(filePath)) { - data = JSON.parse(fs.readFileSync(filePath, 'utf8')); - - // Reset if window expired - if (now - data.windowStart > config.RATE_LIMIT_WINDOW_MS) { - data = { count: 0, windowStart: now }; - } - } - } catch (err) { - // Start fresh on error - data = { count: 0, windowStart: now }; + if (row && now - row.window_start > windowMs) { + d.prepare(`DELETE FROM ${table} WHERE key_hash = ?`).run(keyHash); + row = null; } - // Check if over limit - if (data.count >= config.RATE_LIMIT_MAX_REQUESTS) { - const resetTime = data.windowStart + config.RATE_LIMIT_WINDOW_MS; + if (!row) { + row = { key_hash: keyHash, count: 0, window_start: now }; + } + + if (row.count >= maxRequests) { + const resetTime = row.window_start + windowMs; return { allowed: false, resetTime }; } - // Increment and save - data.count++; - fs.writeFileSync(filePath, JSON.stringify(data, null, 2)); + row.count++; + d.prepare(` + INSERT INTO ${table} (key_hash, count, window_start) + VALUES (?, ?, ?) + ON CONFLICT(key_hash) DO UPDATE SET count=excluded.count, window_start=excluded.window_start + `).run(keyHash, row.count, row.window_start); - return { allowed: true, remaining: config.RATE_LIMIT_MAX_REQUESTS - data.count }; + return { allowed: true, remaining: maxRequests - row.count }; +} + +function _decrementRateLimit(table, keyHash) { + const d = getDb(); + const row = d.prepare(`SELECT * FROM ${table} WHERE key_hash = ?`).get(keyHash); + if (row && row.count > 0) { + d.prepare(`UPDATE ${table} SET count = ? WHERE key_hash = ?`).run(row.count - 1, keyHash); + } +} + +function _pruneExpiredRateLimits(table, windowMs, label) { + const cutoff = Date.now() - windowMs; + const result = getDb().prepare(`DELETE FROM ${table} WHERE window_start < ?`).run(cutoff); + if (result.changes > 0) { + console.log(`Pruned ${result.changes} expired ${label} records`); + } + return result.changes; +} + +function checkRateLimit(email) { + return _checkRateLimit('email_rate_limits', getEmailHash(email), config.RATE_LIMIT_WINDOW_MS, config.RATE_LIMIT_MAX_REQUESTS); } function decrementRateLimit(email) { - const filePath = getRateLimitFilePath(email); - try { - if (fs.existsSync(filePath)) { - const data = JSON.parse(fs.readFileSync(filePath, 'utf8')); - if (data.count > 0) { - data.count--; - fs.writeFileSync(filePath, JSON.stringify(data, null, 2)); - } - } - } catch (err) { - // Ignore errors - } + _decrementRateLimit('email_rate_limits', getEmailHash(email)); } function pruneExpiredRateLimits() { - const now = Date.now(); - const dir = getRateLimitsDir(); - const files = fs.readdirSync(dir); - let pruned = 0; + return _pruneExpiredRateLimits('email_rate_limits', config.RATE_LIMIT_WINDOW_MS, 'rate limit'); +} - for (const file of files) { - const filePath = path.join(dir, file); - try { - const data = JSON.parse(fs.readFileSync(filePath, 'utf8')); - if (now - data.windowStart > config.RATE_LIMIT_WINDOW_MS) { - fs.unlinkSync(filePath); - pruned++; - } - } catch (err) { - // Remove invalid files - fs.unlinkSync(filePath); - pruned++; - } - } +function checkIpRateLimit(ip) { + return _checkRateLimit('ip_rate_limits', getIpHash(ip), config.IP_RATE_LIMIT_WINDOW_MS, config.IP_RATE_LIMIT_MAX_REQUESTS); +} - if (pruned > 0) { - console.log(`Pruned ${pruned} expired rate limit records`); - } - return pruned; +function decrementIpRateLimit(ip) { + _decrementRateLimit('ip_rate_limits', getIpHash(ip)); +} + +function pruneExpiredIpRateLimits() { + return _pruneExpiredRateLimits('ip_rate_limits', config.IP_RATE_LIMIT_WINDOW_MS, 'IP rate limit'); } module.exports = { @@ -295,15 +277,24 @@ module.exports = { deleteAuthRequest, pruneExpiredAuthRequests, - // Rate limiting + // Rate limiting (per-email) checkRateLimit, decrementRateLimit, pruneExpiredRateLimits, + // Rate limiting (per-IP) + checkIpRateLimit, + decrementIpRateLimit, + pruneExpiredIpRateLimits, + // Subscription cache setSubscriptionStatus, getSubscriptionStatus, // Utils ensureDirectories, + closeDatabase, + + // DB access for tests + getDb, }; diff --git a/server/tests/auth.test.js b/server/tests/auth.test.js index 80c0d3d..7bbeead 100644 --- a/server/tests/auth.test.js +++ b/server/tests/auth.test.js @@ -1,4 +1,4 @@ -const { describe, it, before, after, beforeEach } = require('node:test'); +const { describe, it, after, beforeEach } = require('node:test'); const assert = require('node:assert'); const request = require('supertest'); @@ -25,6 +25,7 @@ describe('Auth Routes', () => { cleanTestData(); storage.ensureDirectories(); lastSentEmail = null; + emailService.validateEmail = async () => ({ valid: true }); }); after(() => { @@ -60,6 +61,17 @@ describe('Auth Routes', () => { assert.strictEqual(res.status, 400); }); + it('should reject email with low confidence from SES insights', async () => { + emailService.validateEmail = async () => ({ valid: false, reason: 'Low confidence' }); + + const res = await request(app) + .post('/auth/send-magic-link') + .send({ email: 'bad@disposable.com' }); + + assert.strictEqual(res.status, 400); + assert.ok(res.body.error.includes('invalid')); + }); + it('should rate limit after too many requests', async () => { const email = 'ratelimit@example.com'; diff --git a/server/tests/email.test.js b/server/tests/email.test.js new file mode 100644 index 0000000..6fd0045 --- /dev/null +++ b/server/tests/email.test.js @@ -0,0 +1,77 @@ +const { describe, it, beforeEach, mock } = require('node:test'); +const assert = require('node:assert'); + +// Setup must be required first +require('./setup'); + +describe('Email Service - validateEmail', () => { + let emailModule; + let mockSend; + + beforeEach(() => { + // Clear module cache so we get a fresh instance with mockable client + delete require.cache[require.resolve('../src/services/email')]; + delete require.cache[require.resolve('@aws-sdk/client-sesv2')]; + + // Mock the SESv2 client before requiring email module + const sesv2 = require('@aws-sdk/client-sesv2'); + mockSend = mock.fn(); + const OriginalClient = sesv2.SESv2Client; + sesv2.SESv2Client = class MockSESv2Client extends OriginalClient { + constructor(...args) { + super(...args); + this.send = mockSend; + } + }; + + emailModule = require('../src/services/email'); + }); + + it('should return valid for HIGH confidence verdict', async () => { + mockSend.mock.mockImplementation(async () => ({ + MailboxValidation: { + IsValid: { ConfidenceVerdict: 'HIGH' }, + }, + })); + + const result = await emailModule.validateEmail('good@example.com'); + assert.deepStrictEqual(result, { valid: true }); + }); + + it('should return invalid for LOW confidence verdict', async () => { + mockSend.mock.mockImplementation(async () => ({ + MailboxValidation: { + IsValid: { ConfidenceVerdict: 'LOW' }, + }, + })); + + const result = await emailModule.validateEmail('bad@disposable.com'); + assert.strictEqual(result.valid, false); + assert.ok(result.reason.includes('Low confidence')); + }); + + it('should return valid when MailboxValidation is missing', async () => { + mockSend.mock.mockImplementation(async () => ({})); + + const result = await emailModule.validateEmail('unknown@example.com'); + assert.deepStrictEqual(result, { valid: true }); + }); + + it('should fail open on SES error', async () => { + mockSend.mock.mockImplementation(async () => { + throw new Error('SES unavailable'); + }); + + const result = await emailModule.validateEmail('error@example.com'); + assert.deepStrictEqual(result, { valid: true }); + }); + + it('should fail open on timeout', async () => { + mockSend.mock.mockImplementation(async () => { + throw new DOMException('The operation was aborted', 'AbortError'); + }); + + const result = await emailModule.validateEmail('slow@example.com'); + assert.deepStrictEqual(result, { valid: true }); + }); +}); diff --git a/server/tests/setup.js b/server/tests/setup.js index b85cb97..31bcfed 100644 --- a/server/tests/setup.js +++ b/server/tests/setup.js @@ -1,5 +1,4 @@ const fs = require('fs'); -const path = require('path'); // Set up test environment variables process.env.JWT_SECRET = 'test-secret-key-for-testing-only-32chars'; @@ -15,6 +14,8 @@ process.env.DATA_DIR = './tests/test-data'; const testDataDir = './tests/test-data'; function cleanTestData() { + const storage = require('../src/storage'); + storage.closeDatabase(); try { if (fs.existsSync(testDataDir)) { fs.rmSync(testDataDir, { recursive: true, force: true }); diff --git a/server/tests/storage.test.js b/server/tests/storage.test.js index 4942050..f405e94 100644 --- a/server/tests/storage.test.js +++ b/server/tests/storage.test.js @@ -1,4 +1,4 @@ -const { describe, it, before, after, beforeEach } = require('node:test'); +const { describe, it, after, beforeEach } = require('node:test'); const assert = require('node:assert'); const fs = require('fs'); const path = require('path'); @@ -74,24 +74,14 @@ describe('Storage', () => { const requestId = 'expired-request'; const email = 'test@example.com'; - // Create request with old timestamp - const oldTimestamp = Date.now() - config.REQUEST_ID_EXPIRY_MS - 1000; - const filename = `${oldTimestamp}-${requestId}.json`; - const filePath = path.join(testDataDir, config.AUTH_REQUESTS_DIR, filename); + storage.createAuthRequest(requestId, email); - fs.writeFileSync(filePath, JSON.stringify({ - request_id: requestId, - email: email, - status: 'pending', - created_at: oldTimestamp, - session_token: null, - })); + // Backdate via direct SQL + const oldTimestamp = Date.now() - config.REQUEST_ID_EXPIRY_MS - 1000; + storage.getDb().prepare('UPDATE auth_requests SET created_at = ? WHERE request_id = ?').run(oldTimestamp, requestId); const retrieved = storage.getAuthRequest(requestId); assert.strictEqual(retrieved, null); - - // File should be deleted - assert.strictEqual(fs.existsSync(filePath), false); }); }); @@ -166,11 +156,9 @@ describe('Storage', () => { it('should expire cache entries after 10 seconds', () => { storage.setSubscriptionStatus('ttl@example.com', false, null); - // Manually backdate the entry - const cachePath = path.join(testDataDir, 'subscriptions.json'); - const data = JSON.parse(fs.readFileSync(cachePath, 'utf8')); - data['ttl@example.com'].updatedAt = Date.now() - 11000; - fs.writeFileSync(cachePath, JSON.stringify(data, null, 2)); + // Backdate via direct SQL + const oldTime = Date.now() - 11000; + storage.getDb().prepare('UPDATE subscription_cache SET updated_at = ? WHERE email = ?').run(oldTime, 'ttl@example.com'); const status = storage.getSubscriptionStatus('ttl@example.com'); assert.strictEqual(status, null); @@ -186,11 +174,10 @@ describe('Storage', () => { describe('Pruning', () => { it('should prune expired auth requests', () => { - // Create an expired request + // Create an expired request via createAuthRequest + backdate + storage.createAuthRequest('old-request', 'test@example.com'); const oldTimestamp = Date.now() - config.REQUEST_ID_EXPIRY_MS - 1000; - const filename = `${oldTimestamp}-old-request.json`; - const filePath = path.join(testDataDir, config.AUTH_REQUESTS_DIR, filename); - fs.writeFileSync(filePath, JSON.stringify({ created_at: oldTimestamp })); + storage.getDb().prepare('UPDATE auth_requests SET created_at = ? WHERE request_id = ?').run(oldTimestamp, 'old-request'); // Create a valid request storage.createAuthRequest('valid-request', 'test@example.com'); @@ -198,7 +185,7 @@ describe('Storage', () => { const pruned = storage.pruneExpiredAuthRequests(); assert.strictEqual(pruned, 1); - assert.strictEqual(fs.existsSync(filePath), false); + assert.strictEqual(storage.getAuthRequest('old-request'), null); assert.notStrictEqual(storage.getAuthRequest('valid-request'), null); }); }); diff --git a/src/chrome_manifest.json b/src/chrome_manifest.json index c6c75d2..ba590aa 100644 --- a/src/chrome_manifest.json +++ b/src/chrome_manifest.json @@ -3,7 +3,7 @@ "description": "Spend less time on YouTube. Customize YouTube's user interface to be less engaging.", "homepage_url": "https://github.com/lawrencehook/remove-youtube-suggestions", "manifest_version": 3, - "version": "4.3.72", + "version": "4.3.73", "icons": { "16": "/images/16.png", diff --git a/src/firefox_manifest.json b/src/firefox_manifest.json index 600c9f5..9667c91 100644 --- a/src/firefox_manifest.json +++ b/src/firefox_manifest.json @@ -3,7 +3,7 @@ "description": "Spend less time on YouTube. Customize YouTube's user interface to be less engaging.", "homepage_url": "https://github.com/lawrencehook/remove-youtube-suggestions", "manifest_version": 2, - "version": "4.3.72", + "version": "4.3.73", "icons": { "16": "/images/16.png", diff --git a/src/options/body.css b/src/options/body.css index b4bab7f..4e92c10 100644 --- a/src/options/body.css +++ b/src/options/body.css @@ -1019,6 +1019,13 @@ html[dark_mode='true'] .upgrade-plan[selected] { width: fit-content; } +#upgrade_footer { + font-size: 0.7rem; + color: var(--label-color); + opacity: 0.6; + align-self: flex-end; +} + /* Premium lock icon for options */ html:not([is_premium='true']) .option[data-premium='true'] { opacity: 0.25; diff --git a/src/options/main.html b/src/options/main.html index 1ffff37..40ecdf4 100644 --- a/src/options/main.html +++ b/src/options/main.html @@ -222,6 +222,7 @@ +