Skip to content

Feature/simpler model names#8

Open
m-messer wants to merge 3 commits into
mainfrom
feature/simpler_model_names
Open

Feature/simpler model names#8
m-messer wants to merge 3 commits into
mainfrom
feature/simpler_model_names

Conversation

@m-messer

Copy link
Copy Markdown
Member

Problem: The model parameter required callers to know and supply exact OpenAI model strings (e.g. gpt-4o-mini).

Solution: Introduce a MODEL_ALIASES dict in evaluation.py that maps four human-friendly names (small, medium, large, reasoning) to their current OpenAI model strings. The evaluation function resolves the alias at runtime before any API calls, falling back to the raw string if the value isn't a known alias. This means specific model strings still work unchanged, while common cases are abstracted behind stable names that can be updated in a single place.

Changes:

  • app/evaluation.py — added MODEL_ALIASES dict at module level; evaluation_function now resolves parameters['model'] through the dict before passing it to the three OpenAI API calls
  • app/evaluation_tests.py — updated test model variable from 'gpt-4o-mini' to 'small' so all existing integration tests exercise alias resolution; added TestModelAliases class with unit tests for alias coverage and raw string
    passthrough (no API calls)
  • app/docs/dev.md — replaced model description with alias reference table; updated code examples to use 'small'
  • app/docs/user.md — replaced dated model suggestion with alias table, including a "when to use" guide per tier

@peterbjohnson peterbjohnson left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is good. I have a suggestion but it may not be good so it's for you to consider:

Would it be helpful for admin (in the app) to have the option to define the mapping from the aliases to the real model?

E.g. a parameter SMALL_MODEL could be set by admin as gpt-4o-mini, and later as something else. Without pushing changes to the GitHub repo and deploying them.

Maybe it's a good idea, or maybe it's an abuse.

@m-messer

Copy link
Copy Markdown
Member Author

I think it's a good idea. I'll change it over. It will be useful for when we support more than GPT models.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants