I've been using GPT-2 (with the 117M model) to generate hip-hop lyrics.
First I gathered thousands of lyrics from this website : http://ohhla.com/
Then I trained GPT-2 on it until the loss curve went flat (it took few days).

Then it could generate text like this : hiphop_lyrics.txt.

Here is the trained model.

At various points I also used the conditional prompt to generate hiphop versions of famous songs :

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---