Spaces:
Running
Running
File size: 2,191 Bytes
d1ab62e 15a5288 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 |
---
title: inference-metrics
emoji: 🐳
colorFrom: blue
colorTo: green
sdk: static
pinned: false
---
# LLM Pricing
A tool to fetch and compare LLM pricing and capabilities across multiple providers.
## Data Sources
This tool uses two primary data sources:
1. **HuggingFace Router API** (https://router.huggingface.co/v1/models) - Primary source for model pricing, context length, and capability flags
2. **Provider-specific APIs** - Fallback source for additional metadata and capabilities
The HuggingFace Router API now provides comprehensive data including:
- Pricing (input/output costs per million tokens)
- Context length
- supports_tools flag
- supports_structured_output flag
- Provider status
When data is available from both sources, the HuggingFace Router data takes priority.
## Installation
```bash
bun install
```
## Usage
```bash
# Fetch all models and enrich with provider data
bun run get-metrics.ts
# Skip specific providers
bun run get-metrics.ts --skip-providers novita featherless
# Test performance for models (requires HF_TOKEN)
HF_TOKEN=your_token bun run get-metrics.ts --test-performance
# Test specific number of models
HF_TOKEN=your_token bun run get-metrics.ts --test-performance --test-limit 10
```
## Supported Providers
- **novita** - Full API support
- **sambanova** - Full API support
- **groq** - Full API support
- **featherless** - Full API support
- **together** - Full API support
- **cohere** - Full API support
- **fireworks** - Full API support
- **nebius** - HF Router data only
- **hyperbolic** - HF Router data only
- **cerebras** - HF Router data only
- **nscale** - HF Router data only
## Output Files
- `enriched_models.json` - Complete enriched model data
- `provider_models_raw.json` - Raw provider API responses for debugging
## Environment Variables
Optional API keys for fetching provider-specific data:
- `NOVITA_API_KEY`
- `SAMBANOVA_API_KEY`
- `GROQ_API_KEY`
- `FEATHERLESS_API_KEY`
- `TOGETHER_API_KEY`
- `COHERE_API_KEY`
- `FIREWORKS_API_KEY`
- `HF_TOKEN` - Required for performance testing
This project was created using `bun init` in bun v1.2.4. [Bun](https://bun.sh) is a fast all-in-one JavaScript runtime.
|