openclawd/config
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Add batch article fetching: derstandard articles url1,url2,...

Browse source
This commit is contained in:
Agent 2026-02-03 21:46:50 +00:00
parent b603036509
commit 01aec23f1f
2 changed files with 22 additions and 8 deletions
Download patch file Download diff file Expand all files Collapse all files

3
TOOLS.md
View file

@ -47,6 +47,7 @@ Helper script: `~/bin/derstandard`
```bash ```bash
derstandard items [max] # Title + URL pairs (tab-separated) derstandard items [max] # Title + URL pairs (tab-separated)
derstandard article <url> # Full article content for a specific URL derstandard article <url> # Full article content for a specific URL
derstandard articles <url1>,<url2>,... # Fetch multiple articles (comma-separated)
derstandard urls [max] # Article URLs only (default: 50) derstandard urls [max] # Article URLs only (default: 50)
derstandard titles [max] # Article titles only derstandard titles [max] # Article titles only
derstandard raw [max] # Full RSS XML derstandard raw [max] # Full RSS XML
@ -57,7 +58,7 @@ derstandard raw [max] # Full RSS XML
**Workflow for news briefing:** **Workflow for news briefing:**
1. `derstandard items` → pick interesting titles 1. `derstandard items` → pick interesting titles
2. `derstandard article <url>` → get full content for selected articles 2. `derstandard articles <url1>,<url2>,...` → get full content for selected articles
## Forgejo Git Access ## Forgejo Git Access

19
bin/derstandard
View file

@ -14,7 +14,8 @@ Usage: derstandard <command> [args]
Commands: Commands:
items [max] Title + URL pairs for selection (default: 50) items [max] Title + URL pairs for selection (default: 50)
article <url> Fetch full article content for a specific URL article <url> Fetch single article content
articles <url1,url2,...> Fetch multiple articles (comma-separated)
urls [max] Article URLs only urls [max] Article URLs only
titles [max] Article titles only titles [max] Article titles only
raw [max] Full RSS XML raw [max] Full RSS XML
@ -27,7 +28,7 @@ fetch_feed() {
$CURL "${FIVEFILTERS_URL}/makefulltextfeed.php?url=${encoded_url}&max=${max}&links=preserve&exc=" $CURL "${FIVEFILTERS_URL}/makefulltextfeed.php?url=${encoded_url}&max=${max}&links=preserve&exc="
} }
fetch_article() { fetch_single_article() {
local url="$1" local url="$1"
local encoded_url=$(printf '%s' "$url" | sed 's/:/%3A/g; s/\//%2F/g; s/\?/%3F/g; s/&/%26/g; s/=/%3D/g') local encoded_url=$(printf '%s' "$url" | sed 's/:/%3A/g; s/\//%2F/g; s/\?/%3F/g; s/&/%26/g; s/=/%3D/g')
$CURL "${FIVEFILTERS_URL}/makefulltextfeed.php?url=${encoded_url}&max=1&links=preserve&exc=" | \ $CURL "${FIVEFILTERS_URL}/makefulltextfeed.php?url=${encoded_url}&max=1&links=preserve&exc=" | \
@ -50,7 +51,19 @@ case "${1:-}" in
;; ;;
article) article)
[ -z "${2:-}" ] && { echo "Usage: derstandard article <url>"; exit 1; } [ -z "${2:-}" ] && { echo "Usage: derstandard article <url>"; exit 1; }
fetch_article "$2" fetch_single_article "$2"
;;
articles)
[ -z "${2:-}" ] && { echo "Usage: derstandard articles <url1,url2,...>"; exit 1; }
IFS=',' read -ra URLS <<< "$2"
for url in "${URLS[@]}"; do
# Extract title from URL slug
title=$(echo "$url" | grep -oP '/\d+/\K[^?]+' | tr '-' ' ' | sed 's/.*/\u&/')
echo "=== ${title} ==="
fetch_single_article "$url"
echo ""
echo ""
done
;; ;;
urls) urls)
fetch_feed "${2:-50}" | grep -oP '<link>\K[^<]+' | grep "derstandard.at/story" fetch_feed "${2:-50}" | grep -oP '<link>\K[^<]+' | grep "derstandard.at/story"