Shortened from hobby-horse, from Middle English hoby, hobyn, hobin (“small horse, pony”), from Old French hobi, *haubi, haubby, hobin ("a nag, hobby"; > Modern French aubin, Italian ubino), of Germanic origin: from Old French hober, ober (“to stir, move”), from Old Dutch hobben (“to toss, move up and down”); or from North Germanic origin related to Danish hoppe (“a mare”), Old Swedish hoppa (“a young mare”), North Frisian hoppe (“horse”); both ultimately from Proto-Germanic *huppōną (“to hop”), from Proto-Indo-European *kewb- (“to bend; a bend, joint”). More at hop, hobble.
The meaning of hobby-horse shifted from "small horse, pony" to "child's toy riding horse" to "favorite pastime or avocation" with the connecting notion being "activity that doesn't go anywhere". Possibly originally from a proper name for a horse, a diminutive of Robert or Robin (compare dobbin).
From Old French hobet, from Medieval Latin hobētus, diminutive of hobe.