Skip to content
This repository was archived by the owner on Jul 20, 2026. It is now read-only.

Update evaluate.py - Calculating gt_pass_rate befre evaluating the expression "gt_pass_rate > 0.99" - #12

Closed
lapidshay wants to merge 1 commit into
bigcode-project:mainfrom
lapidshay:main
Closed

lapidshay wants to merge 1 commit into
bigcode-project:mainfrom
lapidshay:main

Conversation

@lapidshay

Copy link
Copy Markdown

Calculating gt_pass_rate befre evaluating the expression "gt_pass_rate > 0.99"

Calculating gt_pass_rate befre evaluating the expression "gt_pass_rate > 0.99".
@terryyz

terryyz commented Jul 1, 2024

Copy link
Copy Markdown
Collaborator

Hi @lapidshay, thanks for the PR! Please note that the gt_pass_rate is calculated on Line 134.

You may have noticed an error occurring when attempting to print the completed evaluation results. This issue has been resolved, though the new version hasn't been released. I'm still checking if there are other issues, but haven't found any so far. I'll do a release shortly :)

@lapidshay

lapidshay commented Jul 1, 2024 •

Copy link
Copy Markdown
Author

Actually I introduced a bug myself, and thought I resolves something real. Sorry for that.

@lapidshay lapidshay closed this Jul 1, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants