The dispute began when two men at the site allegedly whistled at and sexually harassed a woman from the family