[Usage] Llava-Gemma Pretraining + Fine tuning Usage issue and missing Fine tuned projector.bin

[nlpkiddo-2001]

Describe the issue

Issue:
I first pretrained the projector using Clip + Gemma Model and then FIne tuned the Gemma and Projector, but no matter what It is giving in correct outputs, and the loss is revolving around 1-2 in pretraining for projector and 0.4 - 0.7 in fine tuning. I tried without Lora.

Screenshots:

Kindly assist me. I have a similar setup for Gemma as like in this PR .
#1247

Screenshot of fine tuning from wandb

[shan23chen]

is this first gen gemma or second?