- This new platform is an upgraded version of Codalab allowing you to create either competitions or benchmarks
- This getting started tutorial shows you a simple example of how to create a competition (fancier examples can be found here; the full documentation is here)
- Create a Codalab account (if not done yet)
- Download the sample competition bundle and a sample submission
- Do not unzip them.
- a working competition/benchmark
- you can submit submission.zip for playing with
- we have enabled "Auto approve registration requests" so that anyone could join without approval
- From the front page https://www.codabench.org/ top menu, go to the Benchmark > Management
- Click the green “Upload” button at the top right.
- Upload the sample competition bundle => this will create your competition.
- In your competition page, go to the tab “My submissions”
- Submit the sample submission bundle.
- When your submission finishes, go to the Result tab to check it shows up on the leaderboard.
- Click on the Edit gray button at the top the enter the editor
- Change the logo
- Save your changes
- competition.yaml: contains ALL the configurations of your benchmark/competition
- input_data: contains datasets for training models and datasets for testing models (i.e. X_train, y_train, X_test)
- reference_data: answers to training Datasets (i.e. y_test)
- ingestion_program: defines the logic of how to read user submissions, read your data, train the model and inferences outputs
- scoring_program: defines the logic of how to calculate user submissions score
- participate.md: markdown file about how to participate in this benchmark/competition
- terms.md: about the terms and conditions of the benchmark, this will show up when other participants join the benchmark/competition
- wheat.jpg: logo for this benchmark/competition
You are done with this simple tutorial.
Next, check the more advanced tutorial