Transparency is a lack of pigment or color, such that you can see through one feature to the feature beneath. You can think of this as being similar to tinted or stained glass; some light is allowed to pass through and reflect off what's inside. When used right, transparency can help emphasize or de-emphasize features in a map composition. It can also be used to blend two layers to look as if they are one layer.
This recipe demonstrates transparency for both vectors and rasters, so we'll need an example of each. The lakes.shp and elevlid_D782_6.tif layers will work well for demonstration purposes. Load both of these layers in a fresh project, putting lake.shp on top.

elevlid is on top.elevid and the Transparency tab.

This is really a computer graphics thing, but the simplest explanation is that you're telling the computer to combine a percentage of two different layers in the same location instead of the top layer's value covering. Based on the math of the original colors and their transparency, a blended color is calculated for each pixel on the screen.
This doesn't begin to explain all the possible variations of appearance that can be achieved by mixing multiple layers and multiple transparencies, only tinkering can show you this.
One classic example of transparencies is to mix hillshades and airphotos. You can place either layer on top and then adjust the transparency to let the other show through. Generally, you would place the hillshade underneath in this case (but either can work). The end result is a landscape that appears to have 3D relief, but it looks like an airphoto.
Another classic example is to create a mask layer with a hole cut out around the region that you want to emphasize. You now place the mask layer on top. Before adding transparency, it blocks everything but the hole. Then, you slowly add transparency so that you can see surrounding regions, but they are muted and stand out less. For this technique, try a black, gray, or white fill for the mask layer. Each will have a slightly different look.
When styling vectors, you can apply different transparencies to different features in the same layer if you use Rule Based Rendering. Each rule can have a different transparency value and the entire layer can have yet another transparency modifier in the Layer Rendering section.
Lastly, keep in mind that not all output formats handle transparency well. In particular, be careful using color gradients with transparencies when exporting to PDF. Generally, PNG handles transparency, SVG may work or at least allow to you to edit the transparency after export, unlike image formats.