10.3 Will there be horses or other rideable mounts?
Yes, both ridable mounts will re-appear in Oblivion. Unfortunatly, mounted Combat won't be possible.

3.3 Will there be other modes of travel available in Oblivion besides walking?
Horses appear in the game, and they can be used as mounts. There is also a fast travel ability that allows you to click on places you've visited previously and travel there, after the correct amount of time has passed

TCW: "What about the horses? Will they be able to level up and such just like your character? Please elaborate on the features or these beasts."

Pete Hines: "No, they don’t level up. They are a mode of transportation and, in truth, a bit of a status symbol for the player. They aren’t cheap. You’ll find different towns will have different types available and you can have several of them stabled in towns as you travel around. They’re like any other creature, so if you’re riding one and get attacked and you jump off to fight, your horse can be attacked, fight back, run away, be killed, etc."

Xbox.com: Oblivion features mounts. Will these be personally owned creatures, or rented?

Carter: You can purchase or steal a horse, and ride them around the world.

Q: Mounted steeds now make an exciting introduction within Oblivion; can you provide us with a little more information about this feature and how it’s implemented within the game?

A: You will be able to acquire a horse in Oblivion, and ride it around to your heart’s content. Horses grant you a nice speed boost, as well as really help capture that feel of being a knight riding around on horseback. There are several different varieties of horses to get, each with varying stats and look. While in town, you can store your horse in the stables, and pick it up when you’re ready to go. NPCs can use horses as well. If you’re not careful, they may even steal yours!