My grandpa was probably a horrible person and my world view is shifting

My grandpa passed away before I was born. My mom and her siblings always told good stories about him. I met some people who knew him on his side of the family. The stories they told of him weren't great. They talked about how mean he was to my grandma and how my grandma was so sweet and never yelled back at him or anyone. I knew my grandparents divorced but I didn't know he was that bad. They told stories of him under the influence. I always sort of idolized him and missed him even though I never knew him. The stories my mom and others told were just that good. If he were alive he might suck as a grandparent.