It’s customary for the bride’s sister to be maid or matron of honor, and if she has more than one sister, for the role to be shared between them. Regardless of how close you are, it’s probably best to ask your sister to stand with you at the altar, especially if it will cause a family rift if you don’t.
I know several sisters who aren’t very close but they still stood as each other’s maid of honor at each other’s weddings. The bride’s best friend a/k/a the first choice for MOH can be a bridesmaid and still take part in helping the bride shop for her wedding dress, craft various wedding details and plan the shower and bachelorette parties.
A wedding is a family event and it’s important that you try to avoid any arguments or fights. Ask your sister to be your maid of honor and your BFF to be a bridesmaid. She’ll understand.