News-Analyzer

News-Analyzer is the research engine. It turns mixed market, company, filing, on-chain, macroeconomic, and news inputs into an evidence-linked investment assessment. It is a Rust Axum service. Its public HTTP surface is designed for service-to-service use; Trader-BE is the product-facing orchestrator.

The design separates collection, factual extraction, claim formation, analysis, consistency checking, and narrative writing. This makes a final conclusion easier to audit than a single prompt that asks a model to do everything at once.

API surface

Endpoint	Purpose
GET /health	Process health
POST /api/v1/analysis/init	Build and cache a reusable analysis context
GET /api/v1/market-profile/:symbol	Collect lightweight market and fundamental data without an LLM
GET /api/v1/news/preview	Return headlines, deterministic sentiment, categories, topics, and impact markers
POST /api/v1/fact-extraction	Extract atomic facts from supplied or collected sources
GET /api/v1/fact-sources	Inspect normalized factual source material
POST /api/v1/claim-generation	Turn facts into supported investment claims
POST /api/v1/fundamental-analysis	Assess business and financial quality
POST /api/v1/valuation-analysis	Estimate value and uncertainty
POST /api/v1/risk-analysis	Identify, rank, and monitor risks
POST /api/v1/scenario-analysis	Build probability-weighted outcome cases
POST /api/v1/historical-context	Compare the current setup with historical data and similar periods
POST /api/v1/cross-tab-validation	Find contradictions and cap unsupported conclusions
POST /api/v1/investment-overview	Produce the final decision and narrative
GET /api/v1/analysis-runs/:id	Retrieve a persisted run with its stage outputs

Most analysis requests accept a list of assets, but the current context builder uses the first asset. Product callers should send one asset per request.

Analysis context

Initialization is the shared front door for an interactive analysis session. The request includes a symbol, a news lookback that defaults to 2,160 hours, a maximum of 50 articles, and a five-year valuation horizon.

The context contains normalized raw fundamental data, computed metrics, a quantitative valuation, collected articles, extracted facts, and generated claims. News-Analyzer stores it in Redis for one hour and returns the Redis key as the context identifier.

Context keys are deterministic over symbol, lookback, article limit, horizon, and a cache version. Equivalent requests can therefore reuse the same context. Debug mode bypasses normal stage reads and writes where supported, which is useful for investigation but more expensive.

Collection and source routing

The asset resolver determines what kind of instrument the symbol represents and selects useful providers.

For listed companies, data can include Yahoo market information, Finnhub fundamentals, and SEC EDGAR filing metadata and XBRL facts. For cryptocurrencies, CoinGecko supplies market and asset information and DeFiLlama-style on-chain clients supply chain TVL and stablecoin context where a chain can be mapped. Commodity handling includes gold-oriented spot and macro inputs. Provider availability and symbol support determine the actual fields present.

News collection combines NewsAPI, Yahoo Finance search news, and Finnhub company news. Calls run concurrently. Articles are normalized, merged, ordered, filtered, and deduplicated. URL normalization prevents tracking parameters or superficial URL differences from turning one story into several sources.

For thin provider summaries, the service may fetch the source page and extract readable paragraph text. It also looks for Open Graph or Twitter image metadata when the provider did not return an image. These fetches are concurrency-limited and best effort. Paywalls, consent walls, CAPTCHA pages, login prompts, missing pages, and obvious extraction garbage are treated as unusable rather than as evidence.

The service can group related stories using TF-IDF-style lexical similarity or embedding-based similarity, selected by configuration. Clustering prevents a widely syndicated story from being mistaken for many independent events.

Free news preview

The preview path is deliberately cheaper and less authoritative than research analysis. It collects recent articles, labels each with keyword-based positive, neutral, or negative sentiment, derives overall counts and a score, extracts recurring topics, flags likely high-impact items, and returns headline metadata.

No model is called. The output is suitable for discovery and browsing, not for the final investment recommendation. It is cached separately with a default five-minute lifetime.

Fact extraction

Raw provider payloads and articles are converted into labeled source records. The model is instructed to extract small, checkable facts rather than conclusions. A fact includes its identity, content, source references, data type, timing, and confidence-related metadata.

The extractor’s purpose is evidence normalization. It should preserve reported values, dates, and qualifiers and should not silently turn an estimate into an observed result. Source identifiers make later claims traceable to the original provider material.

Fact output is cached for one hour using a hash of the raw data, symbol, model priority, prompt version, and schema version. Changing evidence or an explicit version changes the cache identity.

Claim generation

Claim generation takes extracted facts, not the unstructured internet, as its primary input. It produces investment-relevant statements with direction, importance, confidence, reasoning, caveats, and supporting fact and source identifiers. Unsupported claims should be absent or marked weak.

Claims are the bridge between factual evidence and judgment. For example, a reported revenue value is a fact; a claim that revenue momentum supports the long-term thesis is an interpretation that must cite the relevant facts.

Claim output is cached for one hour using the fact payload and model configuration.

Fundamental analysis

The fundamental stage combines deterministic computed metrics with supported claims. For an equity it can assess financial health, growth, profitability, cash flow, capital structure, and basic valuation quality. For other asset classes, unavailable company-specific fields are explicitly absent and the assessment uses relevant market or network inputs.

The response includes a summary, category assessments, metric values, source coverage, unavailable data, and model-usage information. Deterministic metric computation happens before the narrative assessment so the model is not asked to invent arithmetic.

Valuation analysis

Valuation starts with a quantitative model selected for the available asset and inputs. Equity work can use cash-flow, discount-rate, terminal-growth, and relative-multiple concepts. Cryptocurrency work can use network and on-chain proxies. Commodity work can use macro relationships. The response separates model inputs, scenarios, sensitivity, relative measures, unavailable fields, and confidence.

The model writes an explanation around quantitative output; it does not replace missing numeric inputs with fabricated precision. A margin of safety and intrinsic estimate are meaningful only to the extent supported by model coverage and confidence.

Risk analysis

The risk stage identifies key risks, places them in an impact-versus-likelihood matrix, records evidence, defines thesis breakers, and names monitoring indicators. It also computes a reliability view based on source and data quality.

A thesis breaker is stronger than an ordinary risk: it describes an observable condition under which the investment case should be reconsidered. Monitoring fields turn a static risk list into follow-up questions.

Scenario analysis

Scenario analysis creates optimistic, base, and pessimistic cases with probabilities, drivers, expected values, returns, and narrative explanations. It compares the drivers across cases and computes a return distribution, including expected return, dispersion, and selected percentiles when inputs permit.

Probabilities and return assumptions are validated and normalized. Scenario output should expose asymmetry rather than hide it in one average number.

Historical context

Historical context runs or reuses the core stages and compares the current state with stored daily price history and metric snapshots. It can find similar periods, compare regimes, summarize forward outcomes, identify historical failure cases, and place current metrics into historical percentiles.

Historical evidence is only available when the analyzer database is connected and has sufficient ingested data. The response has explicit unavailable-data structures so a thin history does not look like a confident absence of risk.

Ingestion uses job records to avoid duplicate work, fetches Yahoo daily bars, and upserts price and metric data. Similarity results are persisted and cached for reuse.

Cross-tab validation

Validation checks whether fundamentals, valuation, risks, scenarios, and the draft overview agree. Deterministic rules look for contradictions such as high risk paired with excessive conviction, weak data paired with a strong verdict, scenario outcomes that conflict with valuation, or a margin of safety that does not support the conclusion.

An optional model critic adds semantic findings that simple rules may miss. Findings have severity, affected stages, explanation, and suggested correction. Blocking findings and warnings reduce confidence and can cap the maximum allowed verdict or conviction.

This stage is a control layer. The final narrative is not allowed to simply ignore it.

Investment overview

The overview obtains the required stage outputs, validates them, and applies deterministic backend decision rules. Those rules consider expected return, margin of safety, risk, evidence quality, and adjusted confidence to select the permitted verdict and conviction. The model then writes the headline, thesis, summary, key drivers, counterpoints, and watch list inside those bounds.

The response also contains source information, price and contextual data, variance and trend details, model usage, and an analysis-run identifier. Trader-BE adapts this to its product response and uses confidence below 0.25 as the refund trigger.

LLM calls and fallback

All model-dependent stages call LLM-Wrapper’s OpenAI-compatible chat-completion endpoint. Model priority comes from a JSON configuration file when present and can be overridden by a comma-separated environment value. The service tries models in priority order when a call fails in a way that permits fallback.

Structured schemas are sent with requests so each stage can deserialize a defined response. Usage metadata is collected and passed upward. A model call can still return structurally repaired but semantically weak data; cross-tab validation and confidence rules remain necessary.

Redis lifetimes

Cached material	Default lifetime
Raw source bundle	900 seconds
Extracted facts	3,600 seconds
Generated claims	3,600 seconds
Analysis context	3,600 seconds
Individual analysis tabs	1,800 seconds in the stage implementations
Historical stage	3,600 seconds
Cross-tab validation	1,800 seconds
Investment overview	1,800 seconds
Market profile	3,600 seconds
News preview	300 seconds by configuration

The general ANALYSIS_STAGE_CACHE_TTL setting defaults to 900 seconds and is used when constructing the shared stage cache, while several stages intentionally pass their own explicit lifetime. Operators should use the stage-specific values above when predicting expiry.

PostgreSQL artifacts

The analyzer database stores historical price bars, historical fundamental facts, metric snapshots, similarity runs, ingestion jobs, analysis runs, and named stage outputs. The investment-overview pipeline creates and completes a run and upserts stage output records. Retrieval by run UUID returns the run with its stored stages.

Database absence does not prevent every current-data analysis, but run retrieval and historical comparison are unavailable or degraded. Logs should clearly state this startup mode.

Configuration

Setting	Default	Meaning
LLM_WRAPPER_URL	internal wrapper at port 11435	Model gateway base address
NEWS_API_KEY	empty	NewsAPI access; empty reduces source coverage
FINNHUB_API_KEY	empty	Finnhub access; empty reduces news and fundamental coverage
REDIS_URL	local Redis	Stage and context cache
DATABASE_URL	local analyzer PostgreSQL	Artifacts and historical inputs
PREVIEW_CACHE_TTL	300 seconds	Free preview cache
ANALYSIS_STAGE_CACHE_TTL	900 seconds	Shared stage-cache default
CLUSTER_STRATEGY	tfidf	Lexical or embedding clustering selection
MODEL_PRIORITY_CONFIG	container model-priority path	Optional model-order file
MODEL_PRIORITY	unset	Comma-separated runtime override
RUST_LOG	info	Log filtering

Data-quality principles

Missing values remain missing. Provider text is untrusted input. A source’s repetition is not independent corroboration. Calculations occur in deterministic code where possible. Model output is constrained by schemas but is still treated as fallible. Conclusions carry confidence and source coverage. Unavailable history or fundamentals must reduce certainty rather than produce invented values.