Several months ago Beehaw received a report about CSAM (i.e. Child Sexual Abuse Material). As an admin, I had to investigate this in order to verify and take the next steps. This was the first time in my life that I had ever seen images such as these. Not to go into great detail, but the images were of a very young child performing sexual acts with an adult.
The explicit nature of these images, the gut-wrenching shock and horror, the disgust and helplessness were very overwhelming to me. Those images are burnt into my mind and I would love to get rid of them but I don’t know how or if it is possible. Maybe time will take them out of my mind.
In my strong opinion, Beehaw must seek a platform where NO ONE will ever have to see these types of images. A software platform that makes it nearly impossible for Beehaw to host, in any way, CSAM.
If the other admins want to give their opinions about this, then I am all ears.
I, simply, cannot move forward with the Beehaw project unless this is one of our top priorities when choosing where we are going to go.
A few observations/thoughts.
- There’s an awful lot of posts basically saying “this is a part of the job of moderation” and I don’t think that’s a particularly empathetic or useful observation. I’ve been on the internet and moderating for long enough to have been exposed to a lot of this, but this is not an inevitability. It’s an outcome of the system we’ve designed, of regulation and law that we have, and of not prioritizing this as a problem strongly enough. Being dismissive of an emotional experience and trauma isn’t particularly helpful.
- I’m not technical enough to explain this, but there are technical and legal issues with CSAM and the lemmy platform that we’ve ran into. For one, there’s no automated scanning tools for this kind of content. My understanding is that even implementing or creating said tools would be difficult because of the way pict-rs and rust are storing images in the first place. You cannot turn off image federation, at all. At best, you can clear the content, but doing so may violate CSAM laws depending on the country and reporting requirements. Someone on the technical side can explain better than I can.
- This isn’t a thread to discuss who’s to blame for CSAM. Please cease all discussions fighting about religion in the comments. I will be removing these comments.
You cannot turn off image federation, at all.
This is correct for Lemmy codebase; but a WIP by the pictrs dev and upstream Lemmy itself.
For now, Beehaw users can go to their settings via the website, and uncheck Show images if they’re so inclined. This should prevent all images in posts and comments from loading automatically for you. This does not translate to other instances, front-ends, or apps. Just the main website.
People keep talking about going to another platform. Personally I think a better idea would be to develop lemmy to deal with these issues. This must be a fediverse wide problem. So some discussion with other admins and the developers is probably the way to go on many of these things. Moreover you work with https://opencollective.com/, can they help. Beyond this, especially CSAM, there must be large funding agencies where one could get a grant to get some real professional programming put into this problem. Perhaps we could raise funds ourselves to help with this too.
So frankly I would like to see Beehaw solve the issues with lemmy, rather then just move to some other platform that will have its own issues. The exception may be if the Beehaw people think that being a safe space creates too big a target that you have to leave the Threadiverse to be safe. That to me seems like letting the haters win. It is exactly what they want. My vote will always be to solve the threadiverse issues rather then run away.
Just my feeling. There may be more short term practical issues that take precedence and frankly it is all up to you guys where you want to take this project.
The solution is to use an already existing software product that solves this, like CloudFlare’s CSAM Detection. I know people on the fediverse hate big companies, but they’ve solved this problem already numerous times before. They’re the only ones allowed access to CSAM hashes, lemmy devs and platforms will never get access to the hashes (for good reason).
If you use CloudFlare as your proxy then all your instances traffic gets routed through CF before ever making it to your server. If someone tries to upload CSAM it will immediately be flagged (before ever making it to your server). CloudFlare then quarantines it and automatically files a report with the National Center for Missing and Exploited Children. There’s more to the prices, but the point is that putting it in the lemmy software is not a good solution, especially when industry standard proven solutions already exist. You don’t have to use CF. You can also use solutions from Google, FB, Microsoft, Thorn, etc.
Wait… why is no access to csam hashes a good thing? Wouldn’t it make it easier to detect if hashes were public?! I feel like I’m missing something here…
Ah thanks, hadn’t thought of that!
Question, from what I saw it seems like every CSAM image ever is assigned a new hash. Isnt it unscalable to asign a separate hash for everything? does that mean that most CSAM images were detected before?
Aren’t there ways to automatically scan the content of posts to detect csam and other disturbed content?
Because completely disallowing images isn’t the solution. For once, people can still share those things through links. And secondly, there are people who need images and the like to communicate.
It could be possible to just disable uploading of images on the platform and rely on third parties (such as Imgur etc) who have the tools and experience to tackle this. It would at least solve the issue of images from other federated instances putting on an instance and it’s not the sole responsibility of the admin to sort it.
From my understanding, beehaw still can get these images from other instances, unless they’ve already disabled that feature like lemm.ee did. Anyway, I think this should be a common effort, because it’s not easy to solve and probably costs a lot of money. Fediverse should be free from CSAM.
I’ve been thinking lately that I kind of miss things like IRC where you couldn’t really post pictures in chat. With things like Discord and Slack the off topic channels often devolve into people just sharing random memes they found funny at the time, and not really talking to each other. I’m sure there’s value in that too, but I think it can take up a lot of oxygen in the social space, so I’m not sure it’s always a win. Different formats encourage different ways of interacting with each other, I guess, and it’s interesting!
Hey, Im not a user from this instance, but I feel worried about you OP.
Perhaps take a break, even if you could not erase your memory’s as some people have said please take care of yourself. If it is not restrictive please go to therapy or in case that it is too expensive there is always a free service in a lot of countries.
A software platform that makes it nearly impossible for Beehaw to host, in any way, CSAM.
I hate to say it, but you’ll need to find a text-only platform. Allowing any image uploads opens the door to things like this.
Besides that, if your concern is that no moderator should be exposed to anything like that, well on a text-only site you might have to deal with disguised spam links to gore, scam, etc. You’ll still have to click on links to effectively moderate.
Maybe you should consider if this is a position that you want to put yourself in again. It sounds like this may just not be for you.
This was my immediate thought as well. It’s unfortunate, but there will probably always be people who abuse online platforms like this. It’s totally okay if you’re not up to the task of moderating disturbing content like that — it sounds like it can be a really brutal job. I don’t know what the moderation tools on Lemmy are like, but maybe there’s a way to flag different kinds of moderation concerns for different moderators (so not everybody has to be exposed to this kind of stuff if they’re not comfortable with it). And maybe there could also be a system where if user’s flag the post it can be automatically marked as NSFW and images can be hidden by default so moderators and other users don’t have to be exposed to it without warning (though of course such a system could potentially be abused as well). But beyond that I’m not sure what else you can do, aside from maybe limiting federation.
deleted by creator
As others have suggested, I think temporarily suspending images until you guys can settle on a safe alternative to lemmy is a good idea.
Im sorry you had to see something like this, i hope you are able to seek out some counceling asap, talk to someone about it. Even something like https://www.7cups.com/ might be helpful.
I think temporarily suspending images until you guys can settle on a safe alternative to lemmy is a good idea.
There is no such thing as a safer alternative to Lemmy. It’s very easy to say things like “use tools” to filter these things, but in actuality it’s anything but, it’s way beyond a foss project. (or Reddit for that matter, though they are trying and good gawd, I just remembered something I saw on reddit and have not thought of for years, damn it)
Well true, but I meant more like a forum with limited access (no images or links) until you meet certain requirements etc. So not totally safe, but a bit safer than the current setup
Sorry to hear that mate! That’s one of the biggest reasons I’ve never wanted to move towards IT forensics even though I think I’d enjoy the actual work. But having to regularly sift through the absolute worst humanity has to offer sounds awful.
Hope the immediate pain of it settles as soon as possible!
This might not be what people want but since beehaw is going to leave Lemmy anyway, couldn’t you just completely defederate and run as an isolated instance? Then you’d have control of what her life gets published without having to deal with federated nastiness?
removed by mod
removed by mod
removed by mod
There are many different institutions based on Christianity scaling from the Roman Catholic Church, Church of England, Presbyterian Church of the United States of America to tiny little churches such as small baptist churches and nondenoms.
So if your criticism is specifically with the Roman Catholic Church, fair enough, but the teachings of Christianity aren’t evil at all and if everyone acted how Jesus acted the world would be a far better place
Those images are burnt into my mind and I would love to get rid of them but I don’t know how or if it is possible
I’m very sorry this happened to you, and I wish I could offer you some advice… but that’s the main reason I stopped hosting open community stuff many years ago. I thought I was hardened enough, but nope; between the spam, the “shock imagery” (NSFL gore, CSAM), the doxxing, and toxic users in general… even having some ads was far from making it all worthwhile. There is a reason why “the big ones” like Facebook or Google churn through 3rd world mods who can’t take it for more than a few months before getting burnt out.
I wish I could tell you that you’ll eventually forget what you’ve seen… but I still remember stuff from 30 years ago. Also don’t want to scare you, but it’s not limited to images… some “fanfiction” with text imagery is evil shit that I still can’t forget either.
Nowadays, you can find automated CSAM identification services, like the one run by Microsoft, so if you integrated that, you could err on the side of caution and block any image it marks as even suspicious. This may or may not work in your jurisdiction, with some requiring you to “preserve the proof” and submit it to authorities (plus different jurisdictions having different definitions of what is an what isn’t breaking the law, and laws against swamping them with false positives… so you basically can’t win). This will also do nothing for the NSFL or text based imagery.
A way to “shield yourself” from all of this as an admin, is to go to an encrypted platform where you can’t even see what’s getting posted, so you never run the risk of seeing that kind of content… but then you end up with zero moderation tools, pushing all the burden onto your users, so not suitable for a safe space.
Honestly, I don’t think there is an effective solution for this yet. It’s been a great time
abusing the good will of the admins and modsstaying on Beehaw, but if you can’t find a reasonable compromise… oh well.This post seems highly reactive to me. I’m sorry to hear of you being exposed to such disturbing material, but I fail to see at true connection of that happening and using Lemmy as the platform. I absolutely agree that nobody should have to experience what you did, but I disagree with the platform change proposition.
I don’t know of any software platform where that would not happen.
Even with a text-only platform people can still post URLs to unsafe content.
I think OP is referring to some kind of automated scanner but I’m not sure there are publicly available ones. I guess using them would come at a cost - either computational or $$. And even so, there can be false positives so you would probably still have to check the report anyway someday.
Does that mean a platform that does not allow any images to be uploaded? Or a platform that has better access control and remediation controls?
I’d be willing to consider either and would love your, particular, feedback on this as well.
I think if a platform has image capabilities this is to be expected. I guess the only exception if there are filters that can be used, but this seems unlikely. So I think it is an image vs. no image decision. The other problem with images is they can be attack vectors from a security point of view. Any complex file format can be an attack vector as interpreters of complex file formats often have bugs.
Can you imagine that the large platforms have whole teams of people that have to look at this stuff all day and filter it out. Not sure how that works, but it is probably the reality. Notice R$ never hosted images.
Did you forget to log into your alts or are you unaware of how the edit button functions?
Storage is super cheap, fwiw.
Now be nice. Of course I know about the edit button. The comments were not posted at the same time and generally later editing is discouraged. Nor are long comments or one comment on different topics great.
Why on earth would I have multiple accounts? I am sure people do, but that too is kind of strange behavior and perhaps abusive depending on how they are used.
Edits are not frowned upon unless you’re just editing a post to make someone look bad
The RATE of storage both the increasing and the bandwidth transferring, is the expensive part.
I just want to say, I am so so so sorry you had to see that.
I accidentally saw some CSAM in the 1990s and you are right, it is burnt into your mind. It’s the real limit case of “what has been seen cannot be unseen” - all I could do was learn to avoid accessing those memories.
If you can access counselling for this, that might be a good option. Vicarious trauma is a real phenomenon.
If you can access counselling for this, that might be a good option. Vicarious trauma is a real phenomenon.
Thank you for the advice. I’m not sure that I’ll need counseling but I’m open to it if need be. Time will tell.
Be sure to keep tabs on yourself, sometimes these things can really sneak up on you.
