About task_id

[rattlesnakey]

In the pre-train stage, the sample should use corresponding task router (using task_id param) to get fused prompt embedding, but in the code, I found task_id set to 0 always in https://github.com/Hzfinfdu/MPMP/blob/master/Deep/modeling_bert.py#L548, is that any thing wrong with it ?

by the way, in the inference stage, what task_id number should the new tasks (e.g. unseen task category) use ?

[Hzfinfdu]

Hi, our model implements an inform_model function to get task_id from the batch in this part of our code.
The current task_id is included in the batch here.

In the inference stage, or we say downstream adapting stage, the model re-initialize an only router here(i.e. with only 1 task) and always set the task_id to 0 (and that is why in modeling_bert the task_id is by default 0 so that we do not need to explicitly operate the task_id). Optimizing this router (or along with the prompts) yields a downstream composition of the pretrained prompts.
btw, if we have some correlation prior in between the downstream task and the pretraining tasks, we can also initialize the router with a pretrained router. In this case, the task_id is also 0.