The followers of an unheard-of-god from another land followed omens that led them to you. The omens indicate that you are to be the mate of an avatar (earthly incarnation) of their god. You disagree but are still harassed by the god's followers.
Rivalry: A rival to you or your family cause upheaval forcing you to relocate possibly to a far distant land. (Work with your DM to detirmine the details)
You become involved with criminal elements of society, taking to a life of crime and earning a notorious reputation and bounty. (work with your DM to determine details)