[Tracking issue] General dataset support

[qgallouedec]

The aim is for all trainers to apply the same procedure in their init function:

• if needed, apply the chat template, then
• if needed, tokenize.

Support todo:

Standard dataset

• BCOTrainer
• CPOTrainer
• DPOTrainer
• GKDTrainer (same as SFTTrainer)
• IterativeSFTTrainer
• KTOTrainer
• NashMDTrainer
• OnlineDPOTrainer
• ORPOTrainer
• PPOv2Trainer
• RewardTrainer [RewardTrainer] Tokenize inputs within trainer #2102
• SFTTrainer (via "dataset_text_field", needs refactoring)
• XPOTrainer

Conversational dataset

• BCOTrainer BCOTrainer conversational dataset support #2107
• CPOTrainer
• DPOTrainer Conversational dataset support for DPOTrainer #2131
• GKDTrainer
• IterativeSFTTrainer
• KTOTrainer
• NashMDTrainer Conversational dataset support for Online DPO #2075
• OnlineDPOTrainer Conversational dataset support for Online DPO #2075
• ORPOTrainer
• PPOv2Trainer
• RewardTrainer [RewardTrainer] Tokenize inputs within trainer #2102
• SFTTrainer (yes, via get_formatting_func_from_dataset for now, needs refactoring)
• XPOTrainer Conversational dataset support for Online DPO #2075

Tokenized dataset (to be discussed, do we want this?)

• BCOTrainer
• CPOTrainer
• DPOTrainer
• GKDTrainer
• IterativeSFTTrainer
• KTOTrainer
• NashMDTrainer
• OnlineDPOTrainer
• ORPOTrainer
• PPOv2Trainer
• RewardTrainer
• SFTTrainer (yes, via custom collate_fn and dataset_kwargs={"skip_prepare_dataset": True},, needs refactoring)
• XPOTrainer

Misc

• Update docs/dataset_format.mdx