Output directory is empty in Trainer

Question

With my script the model is correctly trained and the results are printed, but the results directory is empty. How is that? What is lacking? I think I should have the files described in this answer.

training_args = Seq2SeqTrainingArguments(
    output_dir="./results",
    evaluation_strategy="epoch",
    learning_rate=2e-5,
    per_device_train_batch_size=16,
    per_device_eval_batch_size=16,
    weight_decay=0.01,
    save_total_limit=3,
    num_train_epochs=1,
    fp16=False,
)

trainer = Seq2SeqTrainer(
    model=model,
    args=training_args,
    train_dataset=tokenized_dataset["train"],
    eval_dataset=tokenized_dataset["valid"],
    tokenizer=tokenizer,
    data_collator=data_collator,
)

trainer.train()

score 0 · Answer 1 · answered Apr 14 '23 at 14:55

You are not defining any saving strategy here so by default it will use steps and since you are not define save_steps by default it is 500. This means that first save will be after 500 steps. Since you have only one epoch and your batch size is 16. It means that your training set should be of size at least 8000 (500 * 16) to complete 500 steps and save at least once. My guess would be that it is smaller than that. You can play with the different variables to change that.

Output directory is empty in Trainer

1 Answers1