What is a prototype network in few-shot learning?

A prototype network is a method used in few-shot learning to classify new data points when only a small number of labeled examples (the “shots”) are available per class. It works by creating a representative “prototype” for each class, which is typically the average of the feature vectors (embeddings) of the labeled examples in that class. During inference, a new data point is classified by comparing its feature vector to these prototypes using a distance metric, such as Euclidean or cosine distance. The closest prototype determines the predicted class. This approach is efficient and avoids complex architectures, making it practical for scenarios where labeled data is scarce.

The training process involves two key steps. First, an embedding network (like a convolutional neural network) converts raw input data (e.g., images) into feature vectors. Second, prototypes are computed by averaging the embeddings of the labeled examples for each class in a given task. For example, in a 5-way 1-shot task (5 classes, 1 example per class), each prototype is simply the feature vector of the single example. During training, the network is exposed to many randomly sampled tasks (episodes), each with a small support set (labeled examples) and query set (unlabeled test examples). The model optimizes the embedding network to minimize the distance between query examples and their correct class prototypes while maximizing distances to incorrect ones. This encourages the network to learn features that cluster examples of the same class tightly in the embedding space.

Prototype networks are effective due to their simplicity and computational efficiency. They require no complex meta-learning rules or parameter updates during inference, making them fast to deploy. However, their performance heavily depends on the quality of the embedding network and the assumption that class distributions are well-represented by their prototypes. For instance, if a class has high variability (e.g., “dog” breeds with very different appearances), a single prototype might fail to capture this diversity. Extensions like using multiple prototypes per class or dynamic distance metrics can address such limitations. Practical applications include image recognition for rare objects, medical imaging with limited patient data, or adapting chatbots to new intents with minimal examples. By focusing on feature representation rather than complex architectures, prototype networks offer a straightforward yet powerful baseline for few-shot learning.