Hugging Face Transformers in Practice

Task-Specific Pipelines

from transformers import pipeline

# Sentiment analysis
classifier = pipeline("sentiment-analysis")
result = classifier("This product is amazing!")

# Named entity recognition
ner = pipeline("ner", grouped_entities=True)
entities = ner("Apple CEO Tim Cook announced the new iPhone in Cupertino")

# Question answering
qa = pipeline("question-answering")
answer = qa(question="What is the capital?", context="The capital of France is Paris.")

# Zero-shot classification
classifier = pipeline("zero-shot-classification")
result = classifier(
    "I need to fix a bug in my code",
    candidate_labels=["technical", "business", "personal"],
)

Local Model Inference

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

tokenizer = AutoTokenizer.from_pretrained("microsoft/Phi-3-mini-4k-instruct")
model = AutoModelForCausalLM.from_pretrained(
    "microsoft/Phi-3-mini-4k-instruct",
    torch_dtype=torch.float16,
    device_map="auto",
)

inputs = tokenizer("Explain machine learning:", return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Performance Tips

• Use torch_dtype=torch.float16 to halve memory usage
• Enable device_map="auto" for multi-GPU distribution
• Use torch.compile() for faster inference on compatible models
• Batch inputs for throughput optimization

Common Pitfall: Loading Models Inside Request Handlers

Loading a Hugging Face model on every API request is extremely slow — model loading can take 5–30 seconds. Load the model once at application startup and reuse the pipeline object across requests:

# Wrong: loads model on every call
def classify(text):
    classifier = pipeline("sentiment-analysis")  # slow!
    return classifier(text)

# Correct: load once at module level
_classifier = None

def get_classifier():
    global _classifier
    if _classifier is None:
        _classifier = pipeline("sentiment-analysis", device="cuda" if torch.cuda.is_available() else "cpu")
    return _classifier

def classify(text):
    return get_classifier()(text)

Best Practice: Use torch.no_grad() for Inference

During inference you never need gradient computation, which uses extra memory and slows things down. Always wrap inference calls in a no_grad context manager:

with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=200)

This can reduce memory usage by 30–50% and slightly increase inference speed.

Named Product Reference: Microsoft Phi-3

Microsoft's Phi-3-mini (3.8B parameters) and Phi-3-small (7B) are among the best small models available on Hugging Face. They run locally on a MacBook with 8GB RAM using float16, produce GPT-3.5-comparable outputs for many tasks, and are genuinely useful for offline development without API costs. The microsoft/Phi-3-mini-4k-instruct checkpoint is a good starting point for anyone wanting to experiment with local inference.