Skip to content

Sampling hyperparameters are not universal among models #42

Open
@domenic

Description

@domenic

The explainer currently assumes that a model is best controlled by setting its temperature and top-K sampling hyperparameters.

However, these aren't universal among all models. And, various other models expose more. Others to consider are top-P, max tokens (#36), repetition penalty, presence penalty, frequency penalty, and more.

This poses a challenge for creating an interoperable API for which each browser can bring their own model.

One path here is to pick a set and require that every implementation allow control over such hyperparameters. (Possibly including no real control, e.g. a frequency penalty with max = min = 1.)

It would be especially helpful if others interested in implementing the prompt API were able to chime in with their implementation constraints.

Metadata

Metadata

Assignees

No one assigned

    Labels

    interopPotential concerns about interoperability among multiple implementations of the API

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions