Scale AI built a $13 billion company on one insight: AI needs human feedback to work well, and that feedback has to be collected at scale. Their army of data labelers powers AI development across dozens of industries. Surge AI hit $1.2 billion in annual revenue by 2024 doing the same thing.
But here's the problem: data labeling for language models typically pays $20-40 per hour for generalists. For specialized domains like medicine, rates climb to $90-200 per hour. For finance, where evaluating a trading decision requires not just expertise but skin-in-the-game intuition, the economics break down entirely.
OpenAI reportedly pays traders $150 per hour for RLHF feedback on financial tasks. Even at that rate, they struggle to find qualified contributors. The pool of people who can evaluate complex trading decisions is small, and the best ones have better things to do with their time than rate model outputs.
Financial AI can't be trained the same way ChatGPT was trained. The expertise is too rare, the feedback too nuanced, the scale requirements too large. It needs a fundamentally different approach to human feedback.
Why Standard RLHF Doesn't Translate
Standard RLHF for chatbots works like this:
- Show a human two responses to the same prompt
- Human picks which is better
- Train a reward model on these preferences
- Optimize the language model against the reward model
This works because:
- Anyone can judge whether a response is helpful
- Judgments can be made quickly (seconds per comparison)
- The feedback is immediate; no waiting for outcomes
- Workers are cheap and plentiful
Financial RLHF violates every assumption:
- Only experts can judge financial reasoning quality
- Proper evaluation takes time, especially for multi-factor decisions
- Outcomes unfold over hours, days, or weeks
- Expert traders are expensive and rare
You can't just pay people to label financial data. The expertise required is too specialized, the feedback too complex, and the scale requirements too large.
Types of Feedback Financial AI Needs
Effective financial feedback goes beyond "Response A is better than Response B." Multiple feedback types serve different training purposes:
Intent Feedback
What was the trader trying to accomplish?
{
"intent": {
"objective": "scalp_momentum",
"risk_tolerance": "moderate",
"time_horizon": "intraday",
"thesis": "breakout continuation from consolidation"
}
}Intent grounds the decision. The same action can be good or bad depending on intent. A tight stop makes sense for scalping; it's wrong for swing trading.
Reasoning Feedback
Was the analysis sound?
{
"reasoning_quality": {
"market_read": "correct",
"risk_assessment": "slightly_aggressive",
"timing_logic": "solid",
"blind_spots": ["ignored volume divergence"]
}
}This is process feedback, not outcome feedback. A trade can have correct reasoning but lose money (unlucky) or wrong reasoning and make money (lucky).
Decision Feedback
Was the final decision appropriate given the analysis?
{
"decision_assessment": {
"sizing": "appropriate",
"entry": "good",
"stop_placement": "too_tight",
"target_setting": "realistic"
}
}Even with good analysis, execution decisions can be poor. This feedback addresses the translation from understanding to action.
Outcome Commentary
Post-trade assessment with hindsight:
{
"post_mortem": {
"what_went_well": "entry timing was precise",
"what_could_improve": "stopped out before the move",
"luck_vs_skill": "mostly skill, some unlucky timing",
"would_repeat": true
}
}This closes the loop, connecting initial reasoning to ultimate outcome with human interpretation.
Implicit vs. Explicit Feedback
Not all feedback requires conscious labeling. Some of the most valuable feedback is implicit in trader behavior:
Strategy Modifications
When a trader changes their agent's parameters, they're implicitly saying "this configuration is better than the previous one." Every modification is a preference signal.
Manual Interventions
When a trader overrides their agent, they're saying "I know better in this situation." The pattern of when humans intervene teaches the model its own limitations.
Deployment Duration
Strategies that run for months are implicitly approved. Strategies shut down after days failed some human criterion. Duration is a coarse but scalable quality signal.
Capital Allocation
Traders vote with their money. The amount of capital deployed to different strategies is a revealed preference about relative quality.
Implicit feedback scales infinitely. Every action traders take naturally generates signal without requiring extra effort. The challenge is building systems that capture and structure this signal correctly.
Scaling Feedback Globally
The $150/hour bottleneck exists because traditional RLHF treats feedback as a separate activity. Traders stop trading and start labeling. This doesn't scale.
A better model integrates feedback into the trading workflow:
Feedback as Trading Activity
When traders set up their strategies, they're providing intent. When they modify parameters, they're providing preferences. When they comment on trades, they're providing reasoning. No separate labeling step is required.
Incentive Alignment
Traders want to make money. Systems that help them make money naturally elicit engagement and feedback. The incentive to provide quality information aligns with the incentive to trade well.
Global Accessibility
Crypto rails enable participation from anywhere. A trader in Singapore, a quant in London, and a retail participant in São Paulo can all contribute feedback. Geography doesn't limit scale.
Permissionless Participation
No interview, no application, no approval process. Anyone can participate, and the system learns to weight contributions by quality. High-quality traders' feedback counts more, but quantity helps too.
Quality Control at Scale
More feedback isn't automatically better. Scaling requires quality filtering:
Track Record Weighting
Feedback from profitable traders is weighted higher than feedback from losing traders. The system learns whose opinions are predictive.
Consistency Checking
Feedback that contradicts itself or contradicts market outcomes gets downweighted. The system learns to identify noise.
Expertise Classification
Different traders are experts in different domains. A scalper's feedback on scalping trades is more valuable than their feedback on position trades. The system routes feedback appropriately.
Adversarial Detection
Some participants may try to game the system. Statistical methods can identify patterns that suggest manipulation rather than genuine feedback.
From Feedback to Training Signal
Raw feedback must be processed into training signal:
Aggregation
Multiple pieces of feedback about the same decision are combined. Agreement increases confidence; disagreement flags uncertainty.
Structuring
Free-form comments are parsed into structured categories. "The stop was too tight" becomes stop_placement: too_tight.
Grounding
Feedback is linked to specific decision points, market contexts, and outcomes. Disconnected feedback has limited value.
Balancing
Training data must be balanced across scenarios, outcome types, and feedback categories. Naive use of feedback creates biased models.
What This Enables
With scalable human feedback, financial AI can achieve things that outcome-only training cannot:
- Nuanced judgment: Understanding when aggressive is appropriate vs. when conservative is better
- Style adaptation: Different traders have different valid styles; the model can learn from all of them
- Error attribution: Distinguishing bad luck from bad decisions
- Constraint learning: Understanding not just what works, but what's appropriate given various constraints
Financial AI can't be trained with standard RLHF methods. The expertise is too rare, the feedback too complex, and the scale requirements too large. What's needed is infrastructure that captures human judgment naturally, scales globally through aligned incentives, and converts diverse signals into structured training data.