Skip to content

adding a new recipe on post training cr2 for driving captioning#131

Merged
jingyijin2 merged 7 commits intomainfrom
jingyij/av_captioning
Jan 5, 2026
Merged

adding a new recipe on post training cr2 for driving captioning#131
jingyijin2 merged 7 commits intomainfrom
jingyij/av_captioning

Conversation

@jingyijin2
Copy link
Collaborator

Description

Brief description of the changes in this PR.

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Code refactoring
  • Performance improvement

Changes Made

  • Added/updated documentation
  • Added/updated examples
  • Fixed bugs or issues
  • Improved code quality
  • Updated dependencies

Testing

  • I have tested the changes locally
  • Documentation builds successfully
  • Pre-commit hooks pass
  • Examples run without errors
  • Links and references are valid

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

Additional Notes

Any additional information that reviewers should know.

Copilot AI review requested due to automatic review settings January 5, 2026 17:50
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds comprehensive documentation for post-training the Cosmos Reason 2 model for autonomous vehicle (AV) video captioning and visual question answering (VQA), developed in collaboration between Uber and NVIDIA. The recipe demonstrates how to adapt Cosmos Reason 2 for domain-specific AV captioning through targeted supervised fine-tuning.

Key changes include:

  • A detailed post-training recipe with benchmark definition, zero-shot evaluation, data curation, fine-tuning, and re-evaluation sections
  • Supporting assets (video samples, result visualizations) demonstrating training data and evaluation metrics
  • Integration into the documentation structure with updated navigation files

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
docs/recipes/post_training/reason2/video_caption_vqa/post_training.md Main recipe documentation covering workflow, benchmarks, training configuration, and evaluation results
docs/recipes/post_training/reason2/video_caption_vqa/assets/*.mp4 Video assets demonstrating training samples and annotation examples
docs/recipes/post_training/reason2/video_caption_vqa/assets/*.png Visualization charts for BLEU, MCQ-based VQA, and LingoQA evaluation results
docs/recipes/post_training/reason2/video_caption_vqa/assets/*.json Sample annotation file showing structured ground truth format
docs/recipes/post_training/reason2/video_caption_vqa/SUMMARY.md Navigation file linking to the post-training documentation
docs/recipes/post_training/SUMMARY.md Updated table of contents adding the new Reason 2 recipe entry
docs/recipes/inference/reason2/intbot_showcase/inference.md Typo fix in existing documentation (unrelated to main PR purpose)

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot AI review requested due to automatic review settings January 5, 2026 18:02
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot AI review requested due to automatic review settings January 5, 2026 18:14
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated no new comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@jingyijin2 jingyijin2 requested a review from shunzh January 5, 2026 18:15
…ning.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings January 5, 2026 18:15
…ning.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@jingyijin2 jingyijin2 merged commit eabc86f into main Jan 5, 2026
2 checks passed
@jingyijin2 jingyijin2 deleted the jingyij/av_captioning branch January 5, 2026 18:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant