How to use Hugging Face: Transformers explained, install, cache, tokens and history

To use Hugging Face, browse the Hub for a model or dataset, then load it in Python with the Transformers library: a pipeline runs a task in a few lines, and the same library handles fine-tuning. Hugging Face also hosts demos called Spaces and offers paid compute. FineTune Studio is built on Transformers, PEFT and TRL, so I use this stack daily. This page covers what Hugging Face is, how it earns money, how Transformers works, its licence and history, installation, where models are cached, and fixing token errors.

How do I use Hugging Face?

Create a free account, search the Hub for a model suited to your task, read its model card for usage and licence, then run it with the Transformers library or the model's recommended code. For a quick start, use a pipeline.

Also asked as: how to use hugging face

from transformers import pipeline
classify = pipeline("sentiment-analysis")
print(classify("The new hostel rules are clear and fair."))

From there, explore datasets with the datasets library, try models in Spaces demos, fine-tune with Trainer or TRL, and share your own models and datasets. Use access tokens for gated or private models, and check each model's licence before commercial use. Many models also run through other tools, such as Ollama after conversion. Start with small models so downloads and experiments stay fast on a laptop.

What is Hugging Face?

Hugging Face is a company and platform at the centre of open AI: the Hub hosts hundreds of thousands of models, datasets and Spaces demos, and its open-source libraries, such as Transformers, Datasets, Tokenizers, PEFT, TRL and Diffusers, make those models usable.

Also asked as: what is hugging face? complete beginner guide

Think of it as GitHub for machine learning: people publish models with model cards, version them with git, and others download, use and fine-tune them. Companies such as Meta, Google, Mistral and Alibaba release open models there. For learners, Hugging Face also provides free courses on NLP, LLMs, agents and diffusion. It started in 2016 as a chatbot app before pivoting to open-source tools. Its community features, such as discussions on model pages, are often the fastest way to learn a model's quirks.

How does Hugging Face make money?

Hugging Face earns revenue from paid services on top of its free platform: subscriptions such as Pro and Enterprise Hub plans, Inference Endpoints for deploying models on managed infrastructure, paid compute for Spaces, and enterprise contracts with security, support and private hosting.

It has also raised significant venture funding from major technology companies. The model mirrors other open-source businesses: free tools and hosting build a huge community, and organisations pay for convenience, scale, security and support. For individuals and students, nearly everything needed to learn and build is free.

What is a transformer in Hugging Face?

In Hugging Face, "transformer" refers both to the neural network architecture behind modern language and vision models and to the Transformers library that implements these models. The library provides thousands of pretrained transformer models with a common interface.

Also asked as: what is transformer in hugging face

The architecture, introduced in 2017, uses self-attention to process sequences, and variants include encoders like BERT for understanding, decoders like Llama and GPT-style models for generation, and encoder-decoders like T5 for translation and summarisation. In the library, each model has a configuration, tokenizer and model class, loaded with AutoTokenizer and AutoModel classes from a Hub name.

How does Hugging Face Transformers work?

Transformers downloads a model's configuration, weights and tokenizer from the Hub, builds the model in PyTorch, or other supported frameworks, and runs it. The tokenizer converts text to token IDs, the model processes them, and outputs are decoded back to text or labels.

from transformers import AutoTokenizer, AutoModelForCausalLM
tok = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct")
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct")
ids = tok("What is RAG?", return_tensors="pt")
print(tok.decode(model.generate(**ids, max_new_tokens=40)[0]))

Pipelines wrap these steps for common tasks. The library also supports quantised loading, multi-GPU inference and training with Trainer.

Is Hugging Face Transformers open source?

Yes. The Transformers library is open source under the Apache 2.0 licence, developed on GitHub with thousands of contributors. You can use it freely, including commercially, under the licence terms.

Note that models on the Hub have their own licences, separate from the library: some are fully open, such as Apache 2.0 or MIT, while others have custom licences with restrictions, such as acceptable use policies or limits on commercial use. Always check the model card's licence before deploying a model in a product, especially for client work. Some datasets also carry licences that affect how models trained on them may be used.

When did Hugging Face Transformers come out?

The library began in late 2018 as "pytorch-pretrained-bert", a PyTorch port of Google's BERT. It was renamed "pytorch-transformers" in 2019 as more models were added, then "transformers" later that year, with TensorFlow support.

Its paper, "Transformers: State-of-the-Art Natural Language Processing", was presented in 2020. Since then, it has grown to support text, vision, audio and multimodal models, quantisation, and integration with training libraries. Its rapid adoption made Hugging Face the default place to share and use open models. Early versions focused on text models like BERT and GPT-2; the library's unified Auto classes, which load any supported architecture by name, were a major reason it spread so quickly through research and industry.

How do I install the Hugging Face Transformers library?

Install it with pip in a virtual environment, along with PyTorch: pip install transformers torch. Add extras as needed, such as accelerate for multi-GPU and large models, datasets for data, and peft and trl for fine-tuning.

Also asked as: how to install hugging face transformers library

Steps:

• Create and activate a virtual environment.
• Install PyTorch matching your CUDA version from the PyTorch site if you have a GPU.
• Run pip install transformers accelerate.
• Test with a small pipeline example.
• Log in with huggingface-cli login or the newer hf auth login for gated models.

Pin versions in your requirements file, since new releases sometimes change behaviour.

Where does Hugging Face Transformers save models?

By default, downloaded models are cached in ~/.cache/huggingface/hub on Linux and macOS, and in C:\Users\<you>\.cache\huggingface\hub on Windows. Each model is stored once and reused across projects.

Also asked as: where does hugging face's transformers save models

Change the location by setting the HF_HOME environment variable, or HF_HUB_CACHE for the hub cache specifically, useful when your system drive is small. You can also pass cache_dir when loading a model. Use huggingface-cli scan-cache or the newer hf cache commands to see sizes and delete unused models. To save a fine-tuned model elsewhere, use save_pretrained with your chosen folder.

How do I fix a "no token found" error when downloading from Hugging Face?

This error usually appears when downloading gated or private models without authentication. Accept the model's licence on its Hub page, create an access token in your account settings, then log in with the CLI or set the HF_TOKEN environment variable.

Also asked as: how to fix no token found error while downloading hugging face

Steps:

• Visit the model page and accept its terms if it is gated.
• Create a read access token in Hugging Face settings.
• Run huggingface-cli login or hf auth login and paste the token.
• Or set HF_TOKEN in your environment or .env file.
• Retry the download.

Never commit tokens to git. If you use notebooks on shared platforms, store the token in the platform's secrets feature.

Pranjul Rathour

GenAI Engineer · India

GenAI engineer and AI product builder with 2+ years shipping production-grade AI systems: RAG pipelines, fine-tuned LLMs, hybrid retrieval and multi-modal apps across vision, speech and OCR, architected end to end from ingestion to deployment. Leads engineering for SCULT INDIA's 14-member team, founded the 500+ member TechVerse Enclave community and has mentored 200+ students. Three hackathon first prizes: Changethon 2025 (IIT Roorkee), Product Genesis at Vividhotsava 2025 (CSJMU) and BYTEBATTLE (MeetKats).

Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. For campus sessions the inviting institution covers the visit end to end: travel (a First AC train ticket for journeys up to five hours, a flight for anything longer), accommodation, meals, and pickup and drop; the honorarium and workshop fees are negotiable on WhatsApp or on the call, and requests are welcome from institutions that can cover the visit this way. Email pranjulrathour41@gmail.com.

PortfolioInvite me to your campusHire meLinkedInGitHubXInstagramThreadsFacebookBlueskyDev.toBloggerHashnodeSCULTEmail