🧊 Data Cube AI
Your daily AI news, curated by AI.
The open-source alternative to Feedly AI and Google News — a multilingual (8 languages) AI news aggregator that curates tech breakthroughs, investment deals, practical tips, and YouTube videos through a 4.5-stage LLM pipeline.
English | 简体中文 | Deutsch | Français | Español | Português | 日本語 | 한국어
Live Demo · Report Bug · Request Feature
Demo
https://github.com/user-attachments/assets/9dddaaed-e473-4350-97de-0346cacb6660
Why Data Cube AI?
| Problem | Solution |
|---|
| 500+ AI articles published daily — you read 3 | Curated sources scanned automatically, top stories surface |
| Scattered across Hacker News, Reddit, ArXiv, RSS | One unified dashboard with tech, investment, tips |
| English-only or single-language tools | 8 languages (EN, DE, ZH, FR, ES, PT, JA, KO) — resilient LLM translation |
| Paid tools (Feedly AI $18/mo, etc.) | Free & open source, MIT licensed, self-hostable |
| No actionable context | AI Chat to ask questions, AI Reports with one-click export |
Features
| Feature | Description |
|---|
| 📡 | Tech Feed | AI/ML breakthroughs with embedded YouTube videos and impact ratings |
| 💰 | Investment Tracker | Funding rounds, M&A deals, evidence-linked AI Funding Tracker with free CSV export |
| 💡 | Practical Tips | Curated from hand-picked Reddit communities and expert blogs |
| 🌐 | 8 Languages | DE, EN, ZH, FR, ES, PT, JA, KO — EN-native, AI translation pipeline |
| 📅 | Daily + Weekly | Automated daily collection with weekly rollup views |
| 🤖 | AI Chat | Ask questions about the current week's AI news |
| 📊 | AI Reports | One-click streaming report — export to Word, HTML, Markdown, Text, JSON |
| 📧 | Newsletter | Two-step subscribe with 8-language selector — one email per subscriber in their preferred language |
| 🔍 | SEO/GEO | SSR home/week/article/topic/tool pages, JSON-LD, Atom feed (8 langs), Google News Sitemap, llms.txt, dynamic OG images, security headers, keyword-localized metadata in all 8 languages |
| ♿ | Accessible | WCAG-compliant: focus-visible, ARIA, reduced-motion, skip links |
| 📝 | Editorial Standards | Transparent AI methodology, data sources, pipeline documentation |
| 📱 | Mobile-First | Dynamic viewport, safe area insets, touch-optimized navigation |
| 💸 | Monetization | Team landing page with a working contact form, backend developer API and job board endpoints |
Architecture
Frontend (Vercel) Backend (Railway)
┌─────────────────────┐ ┌──────────────────────────────┐
│ Next.js 16 │ REST │ FastAPI + PostgreSQL │
│ React 19 │◄───────────►│ │
│ Tailwind CSS 4 │ API │ 4.5-Stage Pipeline: │
│ Shadcn/ui │ │ 1. Fetch (RSS, HN, YouTube) │
│ │ │ 2. Classify (LLM) │
│ Pages: │ │ 3. Process (LLM, parallel) │
│ • Tech Feed │ │ 3.5 Translate (7 languages) │
│ • Investment Feed │ │ 4. Save to PostgreSQL │
│ • Tips Feed │ │ │
│ • AI Chat │ │ Data Sources: │
│ • AI Reports │ │ • RSS (load_sources()) │
│ • SSR Week/Article │ │ • Hacker News (Algolia) │
│ • Topic/Tool Pages │ │ • YouTube Data API v3 │
└─────────────────────┘ └──────────────────────────────┘
Quick Start
Prerequisites
Frontend
cd ai-information-hub
cp .env.example .env.local # Add your API keys
npm install
npm run dev # http://localhost:3000
Backend
cd ai-hub-backend
python -m venv venv && source venv/bin/activate
pip install -r requirements.txt
cp .env.example .env # Add your API keys
python -m scripts.init_db --migrate-all
uvicorn app.main:app --reload # http://localhost:8000/docs
Run Data Collection
python -m scripts.daily_collect # Today
python -m scripts.daily_collect --date 2026-02-07 # Specific date
python -m scripts.weekly_collect # Current week
python -m scripts.weekly_collect --week 2026-kw06 # Specific week
Tech Stack
| Layer | Technology |
|---|
| Frontend | Next.js 16, React 19, Tailwind CSS 4, Shadcn/ui, TypeScript |
| Backend | FastAPI, SQLAlchemy, Alembic, PostgreSQL |
| LLM Classification | deepseek-v4.1-flash-1017 → qwen3.7-flash → free fallbacks (OpenRouter, paid-first) |
| LLM Processing | deepseek-v4.1-flash-1017 → qwen3.7-flash → free fallbacks (OpenRouter, paid-first) |
| Translation | qwen3.7-flash → deepseek-v4.1-flash-1017 → free fallbacks (OpenRouter, paid-first) |
| AI-news filter | TypeSafe Jev (optional): drops articles that are not about AI before classification |
| Chat & Reports | Chat: qwen3.7-flash → DeepSeek 1017; reports: DeepSeek 1017 → Qwen (OpenRouter) |
| Newsletter | Resend (sending) + Beehiiv (subscribers) |
| Stock Data | Disabled (HTTP 410) pending market-data licensing — see docs/data-rights.md |
| Hosting | Vercel (frontend), Railway (backend + DB), GitHub Actions (cron) |
| Design | Newsreader display font, isometric cube logo, section-specific color accents, staggered animations |
Data Pipeline
The backend processes news through a 4.5-stage pipeline:
| Stage | What happens | Output |
|---|
| 1. Fetch | Collect from RSS, Hacker News, YouTube; filter by period boundaries | ~210 raw items |
| 2. Classify | Optional AI-news filter (TypeSafe Jev) drops articles that are not about AI; the LLM classifies the rest into tech/investment/tips (tips sources skip this) | Categorized pool |
| 3. Process | Parallel LLM processing: EN-native summaries (global-audience voice), trends + AI editorial brief | 30 tech + 21 investment + 15 tips + 5 videos |
| 3.5. Translate | Translate EN → DE, ZH, FR, ES, PT, JA, KO (paid-first chain, JSON validation + small-batch retry) | 7 languages per item |
| 4. Save | Store in PostgreSQL, intersperse videos into tech feed | Database records |
Daily collections produce reduced counts (10 tech, 5 investment, 5 tips, 2 videos).
API Reference
| Endpoint | Method | Description |
|---|
/api/weeks | GET | List periods (weeks with nested days) |
/api/tech/{periodId} | GET | Tech feed with embedded videos |
/api/investment/{periodId} | GET | Primary/Secondary/M&A data |
/api/tips/{periodId} | GET | Curated tips |
/api/trends/{periodId} | GET | Period trend topics for the magazine-style trend index |
/api/videos/{periodId} | GET | YouTube video summaries |
/api/stock/{ticker} | GET | Disabled — 410 (licensing) |
/api/stock/batch/?tickers=AAPL,NVDA | GET | Disabled — 410 (licensing) |
/api/admin/collect | POST | Trigger full data collection |
/api/admin/newsletter | POST | Send newsletter (per-subscriber language; languages whose translations are not ready are held) |
/api/admin/newsletter/diagnose | POST | Diagnostic: env flags, subscriber counts (no addresses), content counts, test email to the required test_email |
/api/admin/newsletter/test-send | POST | Real newsletter render with one-click unsubscribe headers, sent to test_email only |
/api/admin/backfill-translations | POST | Repair translations that are missing, stale or untranslated (period_id or since, dry_run, cheap) |
/api/developer/register | POST | Register for developer API key |
/api/jobs | GET | AI job listings (DACH region) |
/api/newsletter/unsubscribe | POST | One-click unsubscribe with a signed per-subscriber token |
/api/contact | POST | Contact form → email to |
AI disclosure header. Every endpoint that serves AI-written text — /api/tech, /api/investment,
/api/tips, /api/trends, /api/videos and /api/deals — returns an X-AI-Disclosure response header
stating that the summaries are AI-generated and linking /ai-disclosure (spec AD7). It is a header rather
than a body field because these endpoints return an object keyed by language code, where an extra key would
read as a ninth language to any client iterating the keys. Access-Control-Expose-Headers exposes it to
browser JavaScript. Endpoints that serve no AI text — /api/weeks, /api/stock, /api/jobs,
/api/developer, admin, newsletter and contact — deliberately do not carry it.
Period IDs: daily YYYY-MM-DD or weekly YYYY-kwWW
Full API docs available at /docs (Swagger UI) when running the backend.
Environment Variables
OPENROUTER_API_KEY=sk-or-v1-... # For chat & report features
YOUTUBE_API_KEY=AIza... # For video metadata
NEXT_PUBLIC_API_URL=http://localhost:8000/api # Backend URL
DATABASE_URL=postgresql://user:pass@localhost:5432/aihub
OPENROUTER_API_KEY=sk-or-v1-... # For LLM classification & processing
YOUTUBE_API_KEY=AIza... # For video fetching
POLYGON_API_KEY=... # Unused while stock endpoints are disabled (410)
ADMIN_API_KEY=your-secret-key # Protects admin endpoints
RESEND_API_KEY=re_... # Newsletter sending
BEEHIIV_API_KEY=... # Subscriber management
BEEHIIV_PUBLICATION_ID=pub_... # Beehiiv publication
[email protected]
CORS_ORIGINS=["http://localhost:3000"]
SIGNING_SECRET= # One-click unsubscribe tokens (≥ 32 random characters)
CONTACT_INBOX= # Contact form destination address
TYPESAFE_API_KEY= # Optional: AI-news filter before classification (unset = off)
Deployment
Frontend → Vercel — Set environment variables in dashboard. Auto-deploys on push to main.
cd ai-information-hub && vercel --prod
Backend → Railway — Auto-applies Alembic migrations on startup. Daily collection and the morning newsletter run via GitHub Actions (collection after Berlin day-end at 00:17/01:17 UTC; two-hour metadata prefetch reduces short-feed losses).
cd ai-hub-backend && railway up
Project Structure
DataCube-AI-Space/
├── ai-information-hub/ # Frontend (Next.js)
│ ├── app/ # Pages + API routes
│ │ ├── api/chat/ # AI chat endpoint
│ │ ├── api/report/ # AI report generator
│ │ ├── api/subscribe/ # Newsletter signup (Beehiiv, 8 langs)
│ │ ├── api/og/ # Dynamic OG images
│ │ ├── (localized)/[lang]/ # Localized routes; own root layout, lang from the URL
│ │ │ ├── week/ # SSR week pages (SEO)
│ │ │ ├── news/ # SSR article pages (SEO/GEO)
│ │ │ ├── topic/ # Topic hubs with period-scoped story links
│ │ │ └── tools/ # Localized tool landing pages
│ │ ├── feed.xml/ # Atom 1.0 feed (8 languages)
│ │ ├── (site)/about/ # Editorial standards
│ │ ├── (site)/editorial-policy/ # Publishing principles
│ │ ├── (site)/source-methodology/ # Source and curation methodology
│ │ ├── (site)/corrections/ # Corrections policy
│ │ ├── (site)/ai-disclosure/ # AI-use disclosure
│ │ ├── (site)/contact/ # Publisher contact
│ │ ├── (site)/unsubscribe/ # One-click unsubscribe confirm page
│ │ ├── news-sitemap.xml/ # Google News Sitemap
│ │ ├── (site)/for-teams/ # Enterprise landing
│ │ ├── (site)/premium/ # Premium upgrade
│ │ ├── (site)/login/ # Legacy welcome page (gate removed 2026-08)
│ │ └── api/newsletter/unsubscribe/ # RFC 8058 one-click unsubscribe
│ ├── components/ # React components
│ │ ├── feeds/ # Tech, Investment, Tips feeds
│ │ ├── trend-index.tsx # Period trend index and topic entry points
│ │ └── video-embed.tsx # YouTube player
│ ├── lib/ # Utils, types, API client
│ └── middleware.ts # Prefetch 204 + legacy/?lang= redirects (page paths only; never touches responses)
│
├── ai-hub-backend/ # Backend (FastAPI)
│ ├── app/
│ │ ├── models/ # SQLAlchemy models
│ │ │ ├── developer.py # ApiKey model
│ │ │ ├── job.py # JobListing model
│ │ │ └── subscription.py # Subscription model
│ │ ├── routers/ # API endpoints
│ │ │ ├── developer.py # Developer API keys + rate limiting
│ │ │ ├── jobs.py # Job board CRUD
│ │ │ ├── newsletter.py # One-click unsubscribe
│ │ │ └── contact.py # Contact form
│ │ └── services/ # Business logic
│ │ ├── collector.py # 4.5-stage pipeline
│ │ ├── llm_processor.py # LLM processing + resilient translation
│ │ ├── youtube_fetcher.py
│ │ └── newsletter_sender.py # Resend + Beehiiv
│ ├── alembic/ # DB migrations
│ ├── scripts/ # CLI tools (daily/weekly collect)
│ └── Dockerfile
│
├── docs/ # Translated READMEs, brand, SEO/GEO, documentation maintenance
└── LICENSE
Contributing
Contributions are welcome! See the Contributing Guide for details.
Quick version:
- Fork the repository
- Create a feature branch (
git checkout -b feature/amazing-feature)
- Commit your changes (
git commit -m 'Add amazing feature')
- Push to the branch (
git push origin feature/amazing-feature)
- Open a Pull Request
Check out our good first issues to get started.
Star History
License
This project is licensed under the MIT License — see the LICENSE file for details.
If you find this project useful, please consider giving it a ⭐
Live Demo · Report Bug · Request Feature · Discussions
Daily publication verification (2026-10-27)
The GitHub daily workflows use ai-hub-backend/scripts/daily_automation.py (Python standard library only). A collection must reach completed and expose nonempty English content through the public API. empty/failed editions are retried; running editions are polled without a duplicate trigger. Timeouts and unavailable status/content APIs fail rather than imply completion. Newsletter runs require completed collection and validate the response's actual send counts. No content, partial failure, held languages or invalid results fail; an already-sent edition reports an explicit idempotent skip with no new messages. Each workflow serializes its own runs. Backend authentication failures retain their error as failed; a fresh newsletter lock is pending, not proof of a completed send. Regression coverage: test_daily_automation.py, test_collection_failure_reporting.py, test_newsletter_failure_reporting.py, and test_llm_call_resilience.py.
Historical recovery via POST /api/admin/collect/process reuses saved raw articles and videos with the same running/completed/failed status lifecycle as nightly collection. It saves base content before translation and updates translations afterward; it does not refetch or clear raw source rows.
Model policy (2026-10-27): generation/reports use DeepSeek V4.1 Flash 1017; six-language translation/chat use Qwen3.7 Flash; classification uses DeepSeek 1017. Public news chains end with NVIDIA Nemotron 3.5 Super VL Free and Gemma 4 31B Free; chat/report only use the two paid models. Reasoning stays off, provider prices and output are bounded. Set LLM_MODEL_PROFILE=legacy and redeploy to restore the prior paid routing (frontend restores its prior free router). See model routing and rollback.
Korean translation exception: the live financial sample exposed Qwen translating private-company valuation as market capitalization. Korean uses the generation chain (DeepSeek first), including cheap repairs; the other six languages remain Qwen-first.
Classification QA exception: Qwen misclassified an IPO discussion in an independent holdout, so classification uses the DeepSeek generation chain; Qwen remains the paid fallback.
Source acquisition, evidence labels, bounded selection, historical metadata repair and rollback: source integrity.
