I've been using GPT-2 (with the 117M model) to generate hip-hop lyrics.
First I gathered thousands of lyrics from this website : http://ohhla.com/
Then I trained GPT-2 on it until the loss curve went flat (it took few days).

Then it could generate text like this : hiphop_lyrics.txt.

Here is the translated french version.

Here is the trained model.

At various points I also used the conditional prompt to generate hiphop versions of famous songs :
rap 01
---
rap 02
---
rap 03
---
rap 04
---
rap 05
---
rap 06
---
rap 07
---
rap 08
---
rap 09
---
rap 10
---
rap 11
---
rap 12
---
rap 13
---
rap 14
---
rap 15
---
rap 16
---
rap 17
---
rap 18
---
rap 19
---
rap 20
---
rap 21
---
rap 22
---
rap 23
---
rap 24
---
rap 25
---
rap 27
---
rap 28
---
rap 30
---
rap 31
---
rap 32
---
rap 33
---
rap 34
---
rap 35
---
rap 36
---
rap 37
---
rap 38
---
rap 39
---
rap 40
---
rap 41
---
rap 42
---
rap 43
---
rap 44
---
rap 45
---