I've been using GPT-2 (with the 117M model) to generate hip-hop lyrics.
First I gathered thousands of lyrics from this website : http://ohhla.com/
Then I trained GPT-2 on it until the loss curve went flat (it took few days).
Then it could generate text like this : hiphop_lyrics.txt.
Here is the translated french version.
Here is the trained model.
At various points I also used the conditional prompt to generate hiphop versions of famous songs :
---
---
---
---
---
---
---
---
---
---
---
---
---
---
---
---
---
---
---
---
---
---
---
---
---
---
---
---
---
---
---
---
---
---
---
---
---
---
---
---
---
---
---