When I start a FollowerService and watch the OnNewFollowerDetected event, it always starts off by reporting 100 new followers. They're not new followers. I don't think it's setting its cache when the service starts. This means it always considers everyone to be "new" for the first trigger of the event.
This is the version of TwitchLib I am referencing.
<PackageReference Include="TwitchLib" Version="3.0.1" />