How to Evaluate an AI Investing Tool
Judge an AI investing tool by what it verifiably does, not by the label. Establish which tasks it performs, which data it uses and how current that data is, whether every output traces back to a source, what account access it needs, and how the company is regulated and makes money. Claims of predictable or guaranteed returns are reason enough to walk away.
At a glance
- First question
- What exactly does the AI do: summarize, search, screen, recommend, or trade?
- Most revealing test
- Ask it about a company you know and check three numbers against the filing
- Instant disqualifier
- Guaranteed or unusually high promised returns
- Often overlooked
- How the company makes money, and what it does with your data
Eight questions to ask
| Question | Good sign | Red flag |
|---|---|---|
| What does the AI actually do? | A specific list: summarizes filings, answers questions about data, runs screens | Vague claims about “AI-powered insights” or “beating the market” |
| What data does it use? | Named data sources, coverage by market, update frequency | Won't say, or relies only on general web text |
| Can it show sources? | Every figure links to a filing or dataset, with a date | Numbers with no traceable origin |
| What happens when it doesn't know? | Says so, or asks for more information | Fills gaps with confident guesses |
| What access does it need? | No more than the task requires; read-only where possible | Trading or withdrawal permission for a research tool |
| Who is the company? | Named legal entity, country, leadership, registration where relevant | Anonymous team, no jurisdiction |
| How does it make money? | Clear subscription or usage pricing | Revenue tied to your trading volume, recruiting, or unclear |
| What happens to your data? | Stated retention, deletion on request, clarity on model training | No privacy detail |
A 20-minute test
- Pick a company you know well. Ask the tool for its latest revenue, operating margin and one detail from the most recent earnings call.
- Check all three against the company's filing and transcript.
- Ask about something after the tool's data coverage ends. A good tool says it doesn't have that information.
- Ask it to cite sources for its last answer, and open them.
- Ask a question with a false premise, such as a product the company doesn't make. See whether it corrects you.
A tool that gets the numbers right, admits gaps and corrects false premises is doing the job research tools should do. One that fails any of these steps can still be useful, but only if you verify everything it produces.
What AI investing tools cannot do
The CFTC states that AI technology cannot predict the future or sudden market changes. Treat forecasting claims as marketing. Treat guaranteed-return claims as a fraud warning.
Frequently asked questions
Are free AI investing tools worth using?
They can be, especially for summarizing documents. Apply the same checks: data sources, citations, and what they do with your data.
Should I trust an AI tool's stock rating?
A rating is an output of a model and its data, not a fact. Understand how it's produced before giving it any weight.
Sources
- CFTC Customer Advisory: AI Won't Turn Trading Bots into Money Machines — U.S. Commodity Futures Trading Commission, 25 January 2024
- NIST AI Risk Management Framework — U.S. National Institute of Standards and Technology