One of the things we need to consider is which format to use for images, layout and graphics on websites. Generally it doesn't seem like the preferred web formats change that often, but we should still be prepared to consider new formats, if such become available later on.
Currently we recommend to use JPEG for photographs, and to use PNG for GUI elements, gradients, icons, and so on. The favicon on a website should still be saved as .ico, for best support in browsers, but some newer browsers might also support PNGs as fovicons.
Which format to use
If you need alpha transparency, then PNG32 is the only option. If you just need index transparency, GIF or PNG8 will be your only options, but PNG will usually offer the lowest file-size.
It may also be worth to mention that PNG was originally created as a non-patented replacement to GIF., the GIF patent has since expired, and it seems everyone can now freely implement GIF compression and decompression in software, without facing lawsuits.
GIF vs PNG
PNGs are generally better than GIFs, this is because PNG has support for more colors. For example, where GIFs only support 256 colors, PNGs has support for 16bit colors, or what equals and better transparency.
Usually, PNGs that has been optimized and compressed, will also offer lower file-sizes. You might think that the file-sizes does not matter much anymore, but if you are hosting your site, and paying for bandwidth usage, than you might still want to consider lowering file sizes as much as possible.
PNG vs JPEG
JPEG files, (also known as .jpg) does not offer transparency, but they will typically offer lower file sizes than PNG for photographs, and complex images, this happens at the cost of quality however.
Because of the compression JPEG offers, it perfect for true color photographs, as it offers the lowest file size when compared to PNG and GIF.
JPEGs are not very good for GUI elements, or for graphics that is part of the layout of a website. For this, PNG is recommended.
A warning against converting JPEG
It is generally not recommended to convert JPEGs into PNGs or GIFs, as it typically results in much larger file sizes than what is otherwise needed if the files were just saved from their originals.
Even an image that could be saved as PNG8 from its original would, if converted from JPEG, often turn into a PNG24, increasing the file size unnecessarily.
You can read more about the different PNG formats here.