PHBench: A Benchmark for Predicting Startup Series A Funding from Product Hunt Launch Signals

Battle Count: 73

The paper addresses startup funding prediction rather than financial market trading. However, it shares methodological relevance with quantitative finance: imbalanced classification, ensemble methods, feature engineering from structured data, and benchmark design. The VC deal-flow screening application has tangential relevance to alternative data-driven investment strategies. The ML vs LLM comparison on tabular data is broadly relevant to quantitative research workflows.

Moderate complexity: 61 engineered features require domain-specific knowledge of Product Hunt metadata and Crunchbase linkage. The ensemble pipeline (three components with isotonic calibration) is straightforward. Feature engineering involves log transforms, rank bucketing, interaction terms, and topic flags. The main complexity lies in data collection (PH GraphQL API pagination, URL resolution, Crunchbase domain matching with blocklist) and the benchmark infrastructure (private test scoring, leaderboard). LLM evaluation is simple (API calls with structured prompts).

High reproducibility: public training/validation/test feature files, 61 engineered features documented, five-metric evaluation harness, public leaderboard at phbench.com, all code and baseline models on GitHub. Test labels held privately by maintainers (SWE-bench style). Dataset available on request. 144 experiments fully logged in Appendix D. Minor caveats: isotonic calibration fitted on validation set creates a documented artifact; maker follower temporal leakage documented but not fully ablated on test set.