Skip to content

Support customized script to fetch feeds - #101

Merged
WCY-dt merged 6 commits into
mainfrom
copilot/support-custom-script-fetch-feeds
Nov 27, 2025
Merged

WCY-dt merged 6 commits into
mainfrom
copilot/support-custom-script-fetch-feeds

Conversation

Copilot AI commented Nov 27, 2025 •

Copy link
Copy Markdown
Contributor

Description

Adds support for custom scripts as an alternative to RSS URL-based feed fetching. Scripts are stored in {data_dir}/scripts/ and executed during feed refresh, outputting valid RSS/Atom XML. Also includes additional UX improvements for feed management.

Type of Change

  • 🐛 Bug fix (non-breaking change which fixes an issue)
  • ✨ New feature (non-breaking change which adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • 📝 Documentation update
  • 🎨 Style/UI change
  • ♻️ Code refactoring
  • ⚡ Performance improvement
  • ✅ Test addition/update

Related Issues

Fixes #100

Changes Made

Backend:

  • Added script_path field to Feed model with DB migration
  • Created ScriptExecutor supporting Python, Shell, PowerShell, Node.js, Ruby scripts
  • Modified FetchFeed to execute script when script_path is set
  • Added /api/scripts/{dir,open,list} endpoints for script management
  • Added /api/feeds/refresh endpoint for single feed refresh
  • Security: path traversal prevention, 30s execution timeout

Frontend:

  • Feed type selector (URL vs Script) in Add/Edit modals
  • Script dropdown populated from scripts directory
  • FeedsTab shows clickable script path instead of URL for script-based feeds
  • Added "Refresh Feed" option to feed context menu for individual feed updates
  • Fixed context menu positioning to stay within viewport when near edges
  • Added feed list sorting by name, date added, or category in settings
  • Replaced inline script help with link to documentation
  • i18n translations (en/zh)

Documentation & Examples:

  • Created docs/CUSTOM_SCRIPTS.md with comprehensive script authoring guide
  • Added example scripts in scripts/examples/ (Python, Shell, Node.js)

Screenshots

Before

N/A - new feature

After

N/A - desktop app requires full build

Testing

# Backend tests including script executor
go test -v ./internal/feed/... ./internal/utils/...

# Frontend build
cd frontend && npm run build && npm test

Test Configuration

  • OS: Linux
  • Go Version: 1.24
  • Node Version: 18+

Test Steps

  1. Place a script in {data_dir}/scripts/ that outputs RSS XML
  2. Add feed → select "Custom Script" → choose script
  3. Refresh feeds → script executes, articles appear
  4. Right-click a feed → "Refresh Feed" to update individual feed
  5. Open Settings → Feeds tab → use sort buttons to reorder list

Checklist

  • My code follows the project's style guidelines
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published

Additional Notes

Script example (Python):

#!/usr/bin/env python3
print('''<?xml version="1.0"?>
<rss version="2.0">
  <channel>
    <title>My Custom Feed</title>
    <item><title>Article 1</title><link>https://example.com</link></item>
  </channel>
</rss>''')

See docs/CUSTOM_SCRIPTS.md for complete documentation on writing custom scripts.

Breaking Changes

None - existing URL-based feeds continue to work unchanged.


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

@WCY-dt WCY-dt added 📑documentation Improvements or additions to documentation ✨enhancement New feature or request labels Nov 27, 2025
Co-authored-by: WCY-dt <55525165+WCY-dt@users.noreply.github.com>
Comment thread internal/feed/script_executor.go Dismissed
Comment thread internal/feed/script_executor.go Dismissed
Comment thread internal/feed/script_executor.go Dismissed
Comment thread internal/feed/script_executor.go Dismissed
Comment thread internal/feed/script_executor.go Dismissed
Comment thread internal/feed/script_executor.go Dismissed
Co-authored-by: WCY-dt <55525165+WCY-dt@users.noreply.github.com>
Copilot AI changed the title [WIP] Add support for customized script to fetch feeds Support customized script to fetch feeds Nov 27, 2025
Copilot AI requested a review from WCY-dt November 27, 2025 07:17
@WCY-dt

WCY-dt commented Nov 27, 2025

Copy link
Copy Markdown
Member

@copilot 优化:

  1. 为主页左侧的订阅源列表的右键菜单中新增按钮,可以单独更新该订阅源的文章。
  2. 在项目根目录编写单独的md文档,指示如何编写自定义脚本。然后在使用自定义脚本添加feed的界面中,去掉那些提示,改为一个链接,指向GitHub上存储那个md文档的url。
  3. 初次安装软件时,默认提供几个可用的脚本以作示例。
  4. 主页中的几种右键菜单,当元素的当前展示位置较靠下时,应当将右键菜单展示在上方,以防超出界面。
  5. 设置页的feed列表部分,支持按照名称/添加时间/分类排序,请参照惯例设计该功能。

…g features

Co-authored-by: WCY-dt <55525165+WCY-dt@users.noreply.github.com>

Copilot AI commented Nov 27, 2025 •

Copy link
Copy Markdown
Contributor Author

@copilot 优化:

  1. 为主页左侧的订阅源列表的右键菜单中新增按钮,可以单独更新该订阅源的文章。...

Implemented all 5 requested improvements in commit 18858e2:

  1. Feed context menu refresh - Added "Refresh Feed" button to right-click menu for individual feed updates
  2. Script documentation - Created docs/CUSTOM_SCRIPTS.md with detailed instructions; replaced inline help text with documentation link
  3. Example scripts - Added Python, Shell, and Node.js examples in scripts/examples/ directory
  4. Context menu positioning - Fixed menu to adjust position when near viewport edges (bottom/right)
  5. Feed list sorting - Added sorting controls by name/date added/category in settings

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • nonexistent-feed-url-12345.com
    • Triggering command: /tmp/go-build148044354/b248/discovery.test /tmp/go-build148044354/b248/discovery.test -test.testlogfile=/tmp/go-build148044354/b248/testlog.txt -test.paniconexit0 -test.v=true -test.timeout=5m0s (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

- 添加自定义脚本执行功能,支持Python、Shell、PowerShell等脚本
- 改进前端UI:添加modal关闭处理、侧边栏搜索、文章过滤器优化
- 修复国际化问题:统一locale格式为en-US/zh-CN
- 优化feed获取:当标题缺失时从内容自动生成标题
- 改进设置系统:添加默认视图模式设置
- 增强安全性:改进脚本路径验证防止目录遍历攻击
- 修复刷新逻辑:支持并发feed刷新和滚动位置保持
@WCY-dt
WCY-dt marked this pull request as ready for review November 27, 2025 10:47
@WCY-dt
WCY-dt merged commit c1bde83 into main Nov 27, 2025
7 checks passed
@WCY-dt
WCY-dt deleted the copilot/support-custom-script-fetch-feeds branch November 27, 2025 10:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

📑documentation Improvements or additions to documentation ✨enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support customized script to fetch feeds

3 participants