Skip to content

added the generated_caption_mapper - #131

Merged
HYLcool merged 12 commits into
mainfrom
feat/generate_caption_mapper
Jan 11, 2024
Merged

added the generated_caption_mapper#131
HYLcool merged 12 commits into
mainfrom
feat/generate_caption_mapper

Conversation

@yxdyc

@yxdyc yxdyc commented Dec 12, 2023

Copy link
Copy Markdown
Collaborator

as the title says, related feature request issue #116

@yxdyc yxdyc added enhancement New feature or request dj:multimodal issues/PRs about multimodal data processing labels Dec 12, 2023
@yxdyc yxdyc added this to the Multimodal Support milestone Dec 12, 2023
@yxdyc yxdyc self-assigned this Dec 12, 2023
Comment thread data_juicer/ops/mapper/generate_caption_mapper.py Outdated
Comment thread data_juicer/ops/mapper/generate_caption_mapper.py Outdated
Comment thread data_juicer/ops/mapper/generate_caption_mapper.py Outdated
Comment thread data_juicer/ops/mapper/generate_caption_mapper.py
Comment thread data_juicer/ops/mapper/generate_caption_mapper.py
@yxdyc

yxdyc commented Jan 2, 2024

Copy link
Copy Markdown
Collaborator Author

Thanks @zhijianma 's comments! I have modified the codes accordingly, see this commit.

@zhijianma
zhijianma self-requested a review January 3, 2024 01:58
@yxdyc yxdyc linked an issue Jan 3, 2024 that may be closed by this pull request
@HYLcool HYLcool added the dj:op issues/PRs about some specific OPs label Jan 8, 2024
Comment thread tests/ops/mapper/test_generate_caption_mapper.py Outdated
Comment thread tests/ops/mapper/test_generate_caption_mapper.py Outdated
Comment thread data_juicer/utils/mm_utils.py
Comment thread data_juicer/ops/mapper/generate_caption_mapper.py Outdated
@yxdyc

yxdyc commented Jan 9, 2024

Copy link
Copy Markdown
Collaborator Author

Thanks for @HYLcool 's comments. I couldn't spare the time to fix and test these comments within the last few days. Please help me to continue developing this OP.

@HYLcool

HYLcool commented Jan 9, 2024

Copy link
Copy Markdown
Collaborator

The Huggingface model Salesforce/blip2-opt-2.7b is too large (> 15.5GB). The runner provided by GitHub has only 14GB of disk space. So we decide to test this new OP locally and skip unit tests of it in the GitHub Action. @yxdyc @chenhesen

  • local tests for generate_caption_mapper passed

@HYLcool HYLcool self-assigned this Jan 10, 2024
* doc modifications
@HYLcool
HYLcool requested a review from zhijianma January 10, 2024 08:00
@HYLcool

HYLcool commented Jan 10, 2024

Copy link
Copy Markdown
Collaborator

This OP has passed the local tests. Online unit tests on GitHub for this OP are skipped. Please continue to review the latest commits by me. Thanks. @zhijianma @chenhesen

@chenhesen chenhesen left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cool

@HYLcool
HYLcool merged commit 2588004 into main Jan 11, 2024
@HYLcool
HYLcool deleted the feat/generate_caption_mapper branch January 11, 2024 02:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dj:multimodal issues/PRs about multimodal data processing dj:op issues/PRs about some specific OPs enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[MM] add a Mapper to generate captions based on models

4 participants