1. Select Layer → Transparency → Add Alpha Channel
2. Select the background using the Fuzzy Selection Tool (Magic Wand) and press the Del key.

GIF will allow you single-color transparency, while PNG has full alpha channel support.
See also GIMP Watermark removal

1. Select a path with the Fuzzy Select Tool
2. Select menu Select → To Path
3. Select the Flip Tool
4. Select Affect: Path and Flip Type: Vertical
5. Click into the layer with the selection to flip the path
6. Select the Text Tool and type the text
7. Right click into the text box and select Text along Path
8. Select menu Layer → New Layer
9. Select Path to Selection in window Paths - Brushes / Open the paths dialog
10. Select the Pencil Tool and paint the text
11. Select menu Layer → Transform → Arbitrary Rotation, set the center point, and rotate the text
12. Right click on the temporary layer Floating Selection in window Layers - Brushes and select To new Layer
13. Delete any layers not needed anymore

This is for text along a circle in the bottom of the circle. For the text on top of the circle, omit steps 3-5.
See also Text Along Path - GIMP 2.8 Tutorial For Beginners

Resynthesizer allows you to remove watermarks and similar from an image. Download it from the GIMP Plugin Registry or my wiki. Move all files to GIMPPortable\Data\.gimp\plug-ins\.

For an example, see GIMP Watermark removal