While SVGs can be opened, edited, and written in a text editor, there are plenty of applications offering a graphical user interface (GUI) that make authoring complex SVG graphics easier if you come from a graphics editing background. Perhaps the most obvious choice is Adobe's Illustrator (PC/Mac). However, it is expensive for casual users so my own preference is Bohemian Coding's Sketch (Mac only: http://bohemiancoding.com/sketch/). That itself isn't cheap (currently at $99), but it's still the option I would recommend if you use a Mac.

If you use Windows/Linux or are looking for a cheaper option, consider the free and open-source, Inkscape (https://inkscape.org/en/). It's by no means the prettiest tool to work with but it is very capable (if you want any proof, view the Inkscape gallery at https://inkscape.org/en/community/gallery/).

Finally, there are a few online editors. Google has SVG-edit (http://svg-edit.googlecode.com/svn/branches/stable/editor/svg-editor.html). There is also Draw SVG (http://www.drawsvg.org), and Method Draw, an arguably better looking fork of SVG-edit (http://editor.method.ac/).

Save time with SVG icon services

The aforementioned applications all give you the capability to create SVG graphics from scratch. However, if it's icons you're after, you can probably save a lot of time (and for me, get better results) by downloading SVG versions of icons from an online icon service. My personal favorite is http://icomoon.io/ is also great.

To quickly illustrate the benefits of an online icon service, loading the icomoon.io application gives you a searchable library of icons (some free, some paid):

You select the ones you want and then click download. The resultant file contains the icons as SVGs, PNGs, and also SVG symbols for placement in the defs element (remember the defs element is a container element for referenced elements).

To see for yourself, open example_07-02 and you can see the resultant download files after I'd chosen five icons from http://icomoon.io/.