All the deities in Hinduism have vehicles to ride upon. Lord Shiva has An ox, Goddess Durga has a lion, Lord Vishnu mounts on the Garuda. But Lord Ganesha mounts on a mouse, which actually does not look like a mount able to carry him because of its small size. Here is a story which goes behind how a mouse became Lord Ganesha's vehicle.
Lord Indra's Court Gathering
Once all the deities had assembled at Indra's court for discussing some grave matter. Among them, the other celestial beings such as the Gandharvas and the Apsaras were also present. In the atmosphere of seriousness, one Gandharva, named Krounch, was creating disturbance in the whole gathering. While all the others were involved in the discussions, this Gandharva was more interested in gossiping with the Apsaras.
Lord Indra Cursed Krounch
When Lord Vishnu saw this, initially he tried to warn him indirectly. But when Krounch ignored all his warnings,Lord Indra was enraged and cursed him to become a mouse instantly. The Gandharva kept begging pardon, but all in vain.
Being a mouse, he started wandering in the Dev Loka, creating nuisance again. Irritated by this nuisance, Lord Indra told the guards to throw him out of the Dev Loka too. The guards followed Lord Indra's instructions and the mouse was thrown to the earth.
Krounch Reached Rishi Parashar's Ashram
It is said that after falling on the earth this mouse reached the Ashrama of Rishi Parashar. Here too, his nuisance found no end. He would eat up sweets, grains, and other food items and cut clothes and other items of use of the sages who lived there. It was equally difficult to catch this mouse. Tired of him, Rishi Parashar asked Lord Ganesha to find a way in order to get rid of him.
Ganesha Attacked Krounch
Ganesha used his famous weapon, Pasha. It is believed that Pasha looped around Krounch's neck and brought it at Lord Ganesha's feet. The Krounch fell there and was very frightened. As soon as he recovered his senses he started asking Lord Ganesha to forgive him. It is believed that Lord Ganesha has a child's heart and his heart melts soon. He soon forgave him but also warned him that he should not repeat the mistake. The mouse sincerely agreed to follow his orders. But Krounch wanted to get rid of the curse and become a Gandharva again. Seeing that the mouse was not happy, though he had been forgiven, Ganesha got little confused. Upon inquiring he came to know that earlier Krounch was a Gandharva and that he wanted to retrieve his original form again. That is why he was never at peace.
Ganesha Blessed Krounch
However, Lord Ganesha said that he had no means to remove the curse given by the king of Gods, Lord Indra. But he could of course give him a blessing that would compensate for it. The mouse agreed and Lord Ganesha gave him the boon that since Lord Ganesha is worshiped with before all the other Gods and Goddesses, Krounch would also be worshiped along with him. This is how Krounch, a Gandharva became the mount of Lord Ganesha.
However, there was another problem yet to be resolved. Krounch, being a mouse would not be able to carry the weight of Lord Ganesha's body. So he asked Ganesha to become lighter. Lord Ganesha became lighter and fulfilled Krounch's another wish.
According to another story, there was an Asura named Gajmukhasura. He had a boon due to which no weapon could kill him. To destroy this demon, Lord Ganesha removed one of his teeth and attacked Gajmukhasura.. Frightened by the approaching death, the demon became a mouse and tried to escape. However Lord Ganesha's weapon- Pasha looped around the demon's neck and brought him to the feet of Lord. The demon asked for being forgiven. Lord Ganesha, as we know, his heart melts soon, forgave the demon and made him his vehicle.