WordPress does not support a bilingual or multilingual blog out-of-the-box. There are however Plugins developed by the WordPress community which will allow you to create a multilingual blog easily.
Creating a mulitlingual blog is basically installing WordPress in more than one language and letting the Plugin switch between them. This includes installing .mo languages files which most Plugins will require you to do manually. See Installing WordPress in Your Language for details.
The free WPGlobus, Polylang, qTranslate-X, xili-language or Sublanguage plugins are installable on standalone WordPress sites. For multisite WordPress (one website per language), you can try Multisite Language Switcher, Zanto or Multilingual Press or purchase WPML.