Spaces:

---

huggingface

/

HuggingDiscussions

Like 104

Running

Love that I can call DeepSeek R1 directly from the Hub 🔥

from huggingface_hub import InferenceClient

client = InferenceClient(
    provider="together",
    api_key="xxxxxxxxxxxxxxxxxxxxxxxx"
)

messages = [
    {
        "role": "user",
        "content": "What is the capital of France?"
    }
]

completion = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-R1", 
    messages=messages, 
    max_tokens=500
)

print(completion.choices[0].message)

Is it possible to set a monthly payment budget or rate limits for all the external providers? I don't see such options in billings tab. In case a key is or session token is stolen, it can be quite dangerous to my thin wallet:(

@benhaotang you already get spending notifications when crossing important thresholds ($10, $100, $1,000) but we'll add spending limits in the future

@benhaotang you already get spending notifications when crossing important thresholds ($10, $100, $1,000) but we'll add spending limits in the future

Thanks for your quick reply, good to know!

Would be great if you could add Nebius AI Studio to the list :) New inference provider on the market, with the absolute cheapest prices and the highest rate limits...

Could be good to add featherless.ai

TitanML !!

Hi team,

Quick update-- our ASR adapter is ready for review: https://github.com/huggingface/huggingface.js/pull/2680

All 5 adapter tests, types and lint passed again!

The live service and billing endpoint are working, and we have the current SVG icon ready. Could you confirm the next step for enabling org/provider oruk and creating an ASR staging mapping (including the organization-plan requirement)?

Service docs: https://qwen3-asr.oruk.ai/docs

Best,
Nathan

Hi Hugging Face team,

I'm reaching out on behalf of ERON X, a Czech IT infrastructure and GPU hosting provider.

We operate our own GPU infrastructure in the Czech Republic (European Union) and are interested in becoming an official Hugging Face Inference Provider.

Our current infrastructure includes:

• 5 AI models available through our inference infrastructure
• OpenAI-compatible API managed through LiteLLM Proxy
• API key authentication and centralized model routing
• Self-hosted GPU infrastructure under our management
• European hosting and the ability to offer EU-based inference

We are interested in making our inference capacity available to Hugging Face users, with usage-based billing and competitive token pricing.

We would be happy to implement any additional requirements, including model mapping, billing integration, request identification, and provider-specific API compatibility.

Could you please clarify:

1. Are you currently accepting new inference providers?
2. Can a smaller, independent European GPU infrastructure provider join the program?
3. Is an existing OpenAI-compatible LiteLLM endpoint a suitable starting point for integration?
4. What are the technical and commercial requirements for onboarding?

Company: ERON X
Website: https://www.eronx.cz
Location: Czech Republic, European Union

We would be happy to provide additional technical information, model specifications, performance benchmarks, and API testing credentials privately.

Thank you for your time!

Best regards,
ERON X Team

Hi Hugging Face team,

I am Jun, co-founder of Fractalyze, an inference optimization company building ultra-low-cost model APIs. We optimize GPU kernels, memory use, caching, quantization, and serving configurations to reduce the cost of running existing models, with workload-specific quality and performance validation.

As one public example of our engineering work, Qwen-Image 2.1 went from 13.47s to 1.85s versus SGLang BF16 on a single RTX 5090 at 1024x1024, 40 steps, warm batch 1. Mean LPIPS versus BF16 was 0.112 over 200 held-out DrawBench prompts. This is an image-model benchmark, separate from the Gemma API described below. Results and code: https://huggingface.co/Fractalyze/qwen-image21-rtx5090-showcase

Gemma 4 26B is our first API pilot, not the intended limit of our model catalog. After validating this pilot's quality, performance, reliability, and unit economics, we plan to scale capacity and expand the model lineup. Our goal is to translate lower serving costs into the lowest sustainable API prices.

The current pilot runs on one RTX 5090 and supports OpenAI-compatible chat completions, text-only input/output, bearer authentication, non-streaming responses, and tool calling. Streaming and other required integration features can be prepared before production onboarding.

We would like to start onboarding discussions for paid inference distribution. Who should we work with on eligibility and technical integration? Could you also share the provider commercial terms, platform fees, and payout currency, schedule, and minimums? We can coordinate evaluation access privately.

Website: https://www.fractalyze.io/
Contact: jun@fractalyze.io

Thanks,
Jun
Co-founder, Fractalyze

Hi Hugging Face team! I am Kendell Jeffrey, operator of InfraXsky, and I would like to join Inference Providers.

We serve Qwen/Qwen3-30B-A3B (currently only one live provider on the Hub) through an OpenAI-compatible API, with streaming, tool calling and JSON-schema structured output. Benchmarked with sglang.bench_serving at 20 concurrent requests: p99 time-to-first-token is about 3.1 s.

• JS client PR: https://github.com/huggingface/huggingface.js/pull/2695
• Billing endpoint follows your spec (nano-USD, Inference-Id header)
• /v1/models exposes pricing ($0.11 / $0.45 per million tokens) and context length (40,960 tokens)
• Hosted on a dedicated GPU server in Helsinki, Finland
• Privacy policy (no prompt storage, no training): https://kencloudapi.duckdns.org/privacy

Two questions: is a Team plan required for the Model Mapping step, and how are providers paid out for routed requests?

Contact: kendell.jeffrey@gmail.com

Thanks,
Kendell Jeffrey
InfraXsky

Serenity Edge (Substrate AI): request to join Inference Providers

Hi Hugging Face team,

We are Substrate AI (https://substrate.ai), and we would like to register Serenity Edge as an Inference Provider on the Hub.

About Serenity Edge

• An OpenAI-compatible inference API (https://api.serenityedge.ai/v1): chat completions with streaming, tool calling and structured output (response_format).
• It runs on our own infrastructure in Spain, and data stays in the EU.
• Zero data retention: prompts and completions are not stored (the prompt cache lives for less than 24 hours), and API traffic is never used to train, fine-tune or evaluate models.
• Docs: https://docs.serenityedge.ai · Status: https://status.serenityedge.ai

What is already in place, following the provider guide

• An Inference-Id header (UUID) on every response, including streaming and errors.
• A billing endpoint that returns the cost per request ID in nano-USD; costs are available about a second after each request. We will share its URL and our credentials through your onboarding.
• Pricing (USD per million input and output tokens) and context_length in /v1/models.
• We checked our chat models against your validation criteria: time to first streamed token is well under 5 seconds, and tool calling and structured output work.

Models we would map first

• Qwen/Qwen3.8-27B -> qwen3.8-27b (conversational)
• Qwen/Qwen3.6-35B-A3B -> qwen3.6-35b-a3b (conversational)
  More models (embeddings, reranking, OCR) could follow if their tasks fit.

Our Hub organization is SubstrateAILab (Team plan). We would suggest "serenity-edge" as the provider name.

A few questions

1. What are the next steps to enable our provider account server-side?
2. Both chat models are served as NVFP4 quantizations. We plan to state this on our provider page; is there anything else you need in the mapping?
3. We also serve our own model family (Orion) through Edge, without weights on the Hub. Can API-only models be listed, or is the program limited to models with a public Hub repository?
4. How is routed usage settled between Hugging Face and the provider?
5. Could you share the egress IP ranges your router uses? We would exempt them from our per-IP rate limits before going live.

You can also reach us at lharillo@substrate.ai and contact@substrate.ai.

Thank you,
Leandro Harillo
CTO, Substrate AI

MuninCloud (Bogarra Integrated SL, Spain): request to join Inference Providers

Hi Hugging Face team,

We are MuninCloud (https://munincloud.com), the cloud brand of Bogarra Integrated SL, a Spanish company, and we would like to join Inference Providers.

What is live today

• OpenAI-compatible API: https://api-flex-huggingface.munincloud.com/v1 (chat completions, streaming with token usage in streaming and non-streaming responses).
• /v1/models with pricing (USD per million tokens) and context_length: https://api-flex-huggingface.munincloud.com/v1/models
• Model: Qwen/Qwen3-32B served as AWQ 4-bit with vLLM, context window 16,384 tokens, up to 4,096 output tokens. Price: $0.11 / $0.39 per million input / output tokens.
• Dedicated servers in Germany (Hetzner, Falkenstein), inside the EU.
• Zero data retention: prompts and completions are never written to disk or logs and never used for training. Prefix cache is salted per account (measured: no cross-account cache hits).
• An Inference-Id header (UUID) on every response.
• When our engine is busy we answer 429 with Retry-After, never a 5xx.
• Privacy policy: https://munincloud.com/legal/flex/en/privacy · Terms: https://munincloud.com/legal/flex/en/terms

What we will do next, following the provider guide

• Open the huggingface.js PR (OpenAI-compatible conversational helper).
• Implement the billing endpoint (cost per request ID in nano-USD).
• Upgrade our Hub organization AdminMunin (https://huggingface.co/AdminMunin) to the Team plan when the mapping step is enabled.

Questions

1. How and in which currency is routed usage settled with the provider?
2. Is an AWQ 4-bit deployment acceptable mapped to Qwen/Qwen3-32B (we state the quantization openly)?
3. Does your router read /v1/models with our provider key, or without authentication?
4. What are the next steps to enable our provider account server-side?

Contact: javier.bogarra@munincloud.com

Thank you,
The MuninCloud team

Hi Hugging Face team — I’m Gaurav Sharma, founder of SaaS Labs, and we’d like to bring PyAI Hear (automatic speech recognition) and Speak (text-to-speech) to Inference Providers. Hub organization: https://huggingface.co/pyaibyatomsai.

PyAI helps developers add speech input and spoken answers to their existing apps and agents while keeping their chosen LLM and orchestration. Hear supports 29 recognition languages with automatic detection. Speak’s live catalog offers 200 stock voices across 17 language groups. These are supported-language/catalog counts, not a claim of equal measured quality in every language. Our LiveKit and Pipecat integrations are already published on PyPI.

Current direct pricing: Hear $0.001 per audio minute (synchronous nonempty requests round up to started minutes; empty transcripts are free). Speak $0.04 per generated audio minute, using actual output duration. We’d like to agree audio-specific billing and settlement with your team.

A recent bounded production check generated a 2.48-second WAV in 1,294 ms and transcribed the exact sentence in 1,530 ms (JSON) / 2,439 ms (plain text). This is one short English example and full-response timing—not first-audio latency, a percentile benchmark or SLA. PyAI’s default no-training warranty covers customer inputs and outputs; it is separate from retention, and we do not claim zero retention.

API: https://api.pyai.com/v1 (authenticated OpenAI-compatible audio endpoints). Docs: https://docs.pyai.com. Pricing: https://pyai.com/pricing. Privacy: https://pyai.com/legal/privacy.

Could you confirm eligibility and the model-mapping path for these API products before we submit the JS/Python adapters? In particular: are API-only speech products eligible, which Hub model repositories should they map to, and what are the billing precision and organization-plan requirements for qualification? We can then implement the agreed task schemas and billing endpoint. This is an onboarding request; PyAI is not yet a native HF Inference Provider.

Contact: gaurav@pyai.com. Thank you!