You mentioned we could start testing after perplexity<=50.
I noticed you trained the example model for 220k steps with a small corpus(file size 13M).
But even with twitter_large(370M), my perplexity converges very quickly.
It gets to 50 after about 20k steps. And it gives pretty dummy answers.
Should i just train longer?
BTW is there any chance i can use the reinforcement learning?
Tks!
You mentioned we could start testing after perplexity<=50.
I noticed you trained the example model for 220k steps with a small corpus(file size 13M).
But even with twitter_large(370M), my perplexity converges very quickly.
It gets to 50 after about 20k steps. And it gives pretty dummy answers.
Should i just train longer?
BTW is there any chance i can use the reinforcement learning?
Tks!