I hope I am wrong and somebody tell me how to fix this.
- Three Imperial guards that was walking on my path with a prisoner, I saved the prisoner and killed the three guards.
- One of the guard had the Amulet of Mara on him, I took it. (At that point, I had no idea what it was)
- Completed The Book of Love, received Amulet of Mara and Agent of Mara blessing.
- So I went to the girl in Riverwood to ask to marry her, there was no dialogue option at all. (I had completed the quest to bring back something to their shop.)
- Then as I had patched Skyrim with the 1.5, I thought I could try ask Lydia to marry me after buying the house and all. No, there was no such option either.
- Yes, I had the Amulet of Mara on me all the time!