andre/LaWGPT
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

[FIX] Fix bug in finetune.py

Browse Source
This commit is contained in:
herobrine19
2023-05-23 16:57:44 +08:00
parent cf5e7db067
commit 1720dfa670
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
finetune.py
View File

@ -123,7 +123,7 @@ def train(
) )
tokenizer.padding_side = "left" # Allow batched inference tokenizer.padding_side = "left" # Allow batched inference
def tokenize(prompt): def tokenize(prompt, add_eos_token=True):
# there's probably a way to do this with the tokenizer settings # there's probably a way to do this with the tokenizer settings
# but again, gotta move fast # but again, gotta move fast
result = tokenizer( result = tokenizer(