When you go to another player and press F and make them join your group, the bandits can no longer damage you. When you are in any group you are invincible to bandits. This irks me as I like having the red dot on the map of fellow members but have to sacrifice this in order to not be invincible. I have tested this by getting shot by an arrow bandit 10x without wearing any armor and did not lose any health. This seems like a simple fix to me, since wild animals could always damage you regardless if you were in a group or not and bandits can only damage you when you are not in a group.
We've posted an important announcement regarding the current situation and the future of the game: Announcement