HuggingFace scraped all of GitHub in 2025.

Your repos are definitely in here, mine were despite them being deleted halfway into last year.

https://huggingface.co/spaces/HuggingFaceCode/in-the-stack

Look up your username to check and use the buttons to below to let them know you do not want to be in their dataset.

Fuck HuggingFace and fuck AI!

Fucking what?

"Unlike stack-v3-train, the file contents in this bucket are not PII-redacted. They are the raw file contents as crawled from GitHub and may contain emails, credentials, API keys, and other personal data that was public on GitHub. You are responsible for performing PII redaction on any data you derive from stack-v3-full."

No. Go fuck yourself. If you're going to scrape, have the decency to redact it. I fucking hate you. I hope you get fucking investigated, HuggingFace.

This should be illegal. This should be treated as a severe data breach and a failure of privacy on GitHub's part too.

There is so much stuff on GitHub that people unwillingly upload without knowing and now it's being shared without being redacted to the public in AI datasets.

We need to put an end to this, stand against this bullshit. I am actually fucking ballistic. People need to start class-action lawsuits against this kind of bullshit from HF and fine them until they can no longer operate.

No wonder why GitHub was going down constantly, it's because they were fucking letting HF scrape them of 50TB of data.

@mrmasterkeyboard what do ppl use now instead of GitHub?

And how the hell do we be gone to get all the projects off there

Kinda like Google in that I don't see a way to move forward practically without this

@spycrab @mrmasterkeyboard codeberg and self-hosted forgejo are very good options. I don't miss GitHub at all.

@gryphonmyers @spycrab exactly the same here. I do also self-host Forgejo but it's for projects that are private that may get published to Codeberg later.

@spycrab HF provides an opt-out method for getting your projects out of their dataset. No guarantees though to be honest. I'd also consider writing to whoever could help get your stuff out of that dataset, whether it be governmental data control orgs or something else.

I moved to Codeberg in 2025 as well and removed all of my GH repos, Codeberg recently banned cryptocurrency projects and vibecoded projects too.

There's also git.gay and some others out there.

@mrmasterkeyboard @gryphonmyers I'll look into migrating anything I got on GitHub to codeburg.

@mrmasterkeyboard I can imagine this is going to result in GDPR hell for them

@hexaheximal It will. You can easily check what repos are in the dataset and that AI will be trained on it. If a company like Apple can prove their XNU APSL code is being used in a different kernel exactly the same written by an AI on that dataset without proper attribution by checking if their repos were included and diffing their code and the other kernels code, then all hell will break loose. Same could happen with Linux code being stolen and reused by AI violating GPL.

That's how I see it.

@hexaheximal In fact, I can prove this.

If I remember, ChatGPT or a similar model was able to recite the entire first chapter of the first HP book (fuck Rowling btw). Also, someone "wrote" an ExFAT driver for FreeBSD using AI and it was almost exactly the same as the GPL Linux ExFAT driver and was caught in reviews. It will happen. A company will notice and then sue HF for copyright infringement and license violation.

@mrmasterkeyboard bloody hell 🤬

I just filed issues to opt everything of mine out

@mrmasterkeyboard Damn I had 9 in there. Thanks a bunch!

@mok0 np :D

@jaredwhite @mrmasterkeyboard I am NOT in the stack. Because the day MS bought the SlopHub I could see the writing on the wall and left for pastures greener.

@mrmasterkeyboard
Only public repos or private ones too?

@notsoloud It'd be everything public, but considering that GitHub and HF are kind of buddies I wouldn't be surprised if GitHub secretly let private repos slip in too. Bit of a conspiracy theory, I know, but this is the AI sphere we're talking about.

@mrmasterkeyboard What the actual fuck? I opted out of their previous iteration of this 2 years ago - why are they not respecting those previous opt outs?

@mrmasterkeyboard wtf, the audacity!

"Hey we inhaled all of your code without consent, now you're responsible for cleaning it up"

Maybe a solar flare is the way to go?

@mrmasterkeyboard

This should be illegal

This is illegal, i’m fairly certain. it’s like, super illegal in the EU. the GDPR is all about handling personal information, and they definitely can’t scrape this and then just publish it without any form of consent.

@sodiboo i fucking knew this might be the case (especially with the emails on git commits too) but i wasn't sure. might also be the same here in the UK.

@mrmasterkeyboard these assholes scrapped my repository. I left my work online to help people taking the same examination not for a ghoulish company to pillage.

@mrmasterkeyboard
They can have my crappy code. I'm not playing their "you can opt-out" bs game.

@mrmasterkeyboard thank you!!! Opt out request done.

@mrmasterkeyboard Knowing my code, I know that any AI trained on this data will be worse than one without it.

@mrmasterkeyboard
Damn, I have one in there too. Bailed out start '25 I think, but still... Deleted my user as I would never return.
But, seeing as they still require you to login to GitHub, I don't know how to opt-out from this...

@mrmasterkeyboard 🙏

@mrmasterkeyboard @spycrab Does Codeberg block Hugging Face from scraping public repositories?

@mrmasterkeyboard

I moved everything but 1 repo last year to Codeberg. The one remaining was a clone of someone else's original, and never touched...

-BUT-

Now that it's showed up as 'in the stack', that means there's one more way to bite the same apple multiple times.

I've no clue if the author opted out or is even aware of the scrape. If he DID opt out, #HuggingAss still scraped the clone, and I suspect every original repo is ripe to steal as long as one clone opt-out is left unanswered.

@mrmasterkeyboard truly deranged position from FaceHugger.... Wtf

@mrmasterkeyboard @catsalad thanks, done

@mrmasterkeyboard
You can also report abuse to GH (I just did instead of agreeing to their opt-out flow). I don’t expect more from it than from the opt-out proces the but it feels better.
@viq

@marcink @viq If GH allowed HF to scrape them of 50TB of data, I doubt GH will do anything about it...

@ramsey @spycrab iirc, Codeberg uses iocaine or some sort of anti-AI scraper tool that is actually a poisoner too. you can see a preview here!

https://codeberg.org/evi-editor/evi/!~codeberger~!/

@pixxl yeah, I went absolutely ballistic when I looked into this.

@mrmasterkeyboard the tech industry hates the whole notion of consent, that’s why we have opt-out, dark patterns, proprietary software, genai, remote attestation, planned obsolesence, and all of this other abusive bs

@mrmasterkeyboard If they have my code, I probably am more sabotage than anything else.

@mrmasterkeyboard content warning pleasseeeeeeeeeee

"Yes, there is code from 78 repositories in The Stack v3:"

:(

https://github.com/bigcode-project/opt-out-v2/issues/5330

It's old stuff, but still ...

CC: @mrmasterkeyboard@mastodon.social

@mrmasterkeyboard Sure. Give me write access to your database I'll clean that right up for you.

@mrmasterkeyboard jokes on them, my code is awful

@mrmasterkeyboard Makes you wonder what has already been taken from your work since this was done in 2025. Was the code also in a previous drop and basically already assimilated?

I've been "off" of GitHub for a few years, and finally a motivated to wipe my repos. I wanted to give back...but not like this.

@mrmasterkeyboard And the way you opt out is create a github issue on their repo. Which means you need to log into github to opt out. If you've forgotten you GitHub password or can't pass 2-factor auth because you no longer have access to the email address you gave when signing up there doesn't appear to be a way to opt out.

@mrmasterkeyboard, great. The fucking thing is so long it exceeds request size.

@SubductionRheology @mrmasterkeyboard also creates a way to harass people who "opt out"

@ramsey

#Codeberg tries to stop AI scrapers from knocking it over via aggressive scraping, but it's a constant game of cat and mouse. AI companies know they're not welcome, so they route their traffic through residential proxies to make it hard to detect and block.

https://krebsonsecurity.com/tag/residential-proxy/

@mrmasterkeyboard @spycrab

@mrmasterkeyboard

I did the opt out, but I shouldn’t have to opt out of every scraper. Everything I put on GitHub has a clear license. These licenses all require attribution. Unless your ML system is built to provide attribution on all outputs that are influenced by my works in the inputs, you are not complying with the license.

Having them in a dataset like this is fine (redistribution that does not remove the license text is permitted), it’s just using them as this dataset is intended that I have problems with.

@mrmasterkeyboard @sodiboo
It‘s also super illegal with regards to copyright law basically everywhere. They removed repositories with known Copyleft licenses (like torvalds/linux), but kept those without any, probably betting that those people don‘t know any better about their rights.

@oz1tmm @mrmasterkeyboard

Same here. Four repos from my deleted account. There must be another way, a support email perhaps.

@mrmasterkeyboard my cursed rust macros are mine only.

@mrmasterkeyboard Requested the removal of my repos, but I wonder if they really honor these requests...

@oz1tmm @mrmasterkeyboard
Send a DMCA takedown request to mailto:dmca@huggingface.co.

@lumi @mrmasterkeyboard They do hate consent, don't they... Look at how techbros behave and talk about women.
It is very much consistent behaviour. *sigh* what a world we live in...

@mrmasterkeyboard Wow. Fuck them.

@lumi @mrmasterkeyboard

proprietary software

It might seem counterintuitive, but (apart from regulating such behavior with laws) this is actually a way to prevent such scraping - keeping an "All rights reserved" approach and licensing, and sharing code (as well as the access to private Git remotes) with only trusted creatures, both when it comes to consent and the security of their machines, so the code isn't leaked.

In other words: nobody can scrape something they can't see in the first place.

@aronowski @lumi i have genuinely considered making all of my software proprietary because im sick of this shit we have to live through. idk if i'll go through with it tho.

the term "open-source" is dead anyway and is dated for the world as of current

@mrmasterkeyboard @lumi I have a feeling that the word "proprietary" simply has initial bad assumptions about its meaning. It's not only about big corporate-driven binaries sold commercially, but can also mean a project that's source-available on a private Git remote, and shared only with the creatures, who the owner agrees to share with, e.g. a cooperative of queer, furry, therian hackers, and no normies or companies can access it from the outside.

@aronowski @lumi I used to hate the word proprietary too. I think it has too much negativity around it. There should probably be a better term for this kind of thing that isn't associated with corpos.

@mrmasterkeyboard @aronowski i believe proprietary software is inherently a bad thing, even if non-corpo

the term “open source” was also co-opted by corporations, hence i tend not to use it

what we want, in my opinion, is software that is developed by a community, for a community. free software in a broader sense of the word than what the fsf states

@lumi @aronowski tbh fuck gnu and fuck fsf

eff and fsfe are way better

@mrmasterkeyboard @aronowski i want a new free software definition, one that is broader. yes source code under a license allowing you to have the 4 freedoms, but also with an inclusive community. also rejecting genai. also anti-capitalist. etc

@lumi "Free as in free beer"

@Alsvid free as in free gock

@lumi hell yes thank god

@Alsvid thank gock

@lumi thock

@Alsvid thanks gock

@lumi
Tbh the libre software only says that all the stuff must be available to the users, and doesn't promise anything to anyone else.

I keep thinking that maybe that's the way to go. Promote users into actual members of a common, make them have a say, and share it all with them. Pick them like you pick members of your community. Outside, code can be shared to trusted entities but that's it

@mrmasterkeyboard @aronowski

@lumi
From here, the license of the code doesn't really matter: all that matters is the community, and whatever means they choose for themselves. I don't think a new licence is really needed, instead the focus should be on the people, their needs, their means, their organization.

Licensing code is a trick to make that code packageable (and thus exploitable as a goods) as an entity regardless of its surroundings, how it's used, etc...

@aronowski @mrmasterkeyboard

@rakoo @aronowski @mrmasterkeyboard this could be a last resort, but i do feel having everything done in the open is a big strength; software freedom is very good to have

@lumi
True, maybe there's another layer of including only devs who are not assholes, that circle could be much larger than communities while still keeping white masculinity out

@mrmasterkeyboard @aronowski

@mrmasterkeyboard @lumi @aronowski fsfe is using the same free software definition as the fsf.

I used to not like that the fsfe used github for reuse-tool (it was also on git.fsfe.org, but you can't get an account there without being a member and an account is required in order to open issues or PRs), but it seems that they've moved to codeberg since then so that is no longer an issue.

@noisytoot @aronowski @lumi i just like fsfe more because they don't like RMS

@rakoo @aronowski @mrmasterkeyboard @lumi

Outside, code can be shared to trusted entities but that’s it

Free software cannot have restrictions on distribution. If this is enforced by license, then this is proprietary software. If it’s just everyone with whom source is shared simply choosing not to publish it (even though they have the right to), that’s fine, but that does not scale. With a large enough group of people having access, someone will likely eventually exercise their right to distribute it publicly.

Also, I think this is definitely not the way to go, even as a last resort. All software should be free and everyone has the right to software freedom. The eventual goal of the free software movement is the complete elimination of proprietary software and software freedom for all. If most free software was distributed only to trusted entities and not publicly available on the internet, that would significantly raise the barrier to entry for new people, who would most likely simply continue using proprietary software and maybe never even find out about free software. Widespread adoption of free software is absolutely essential in order to succeed in this goal, and for this to happen it must be publicly available with minimum barriers to entry.

I think non-public free software is fine in limited cases (perhaps when it’s not ready to be published yet), but the right to distribute it further without restriction must be preserved (or it is not free software), and if all free software was distributed like that it would be extremely harmful to the free software movement.

@noisytoot @rakoo @aronowski @mrmasterkeyboard @lumi

According to RMS, it's Free Software is the user has all four rights. Not the general public. See nuclear submarine software for the example I think he uses.

As for restrictions. Wait till I tell you about #PayForFreeSoftware that'll really make your eyes pop. Probably not through licenses though, that stuffs overblown and besides Microsoft knows real control comes from channel control.

@doctormo @rakoo @aronowski @mrmasterkeyboard @lumi I know that the free software definition does not require publication (and in fact licenses that require modifications to be published are non-free, notably the Sybase Open Watcom Public License (which is actually considered open source by the OSI and is one of the few cases where the free software definition and open source definition (or more specifically the FSF and OSI’s interpretation of their respective definitions) differ, the other being the NASA Open Source Agreement, although that is irrelevant)), but nevertheless, if there was no publicly available free software, this would practically kill the free software movement by raising the barrier to entry too high and making it likely that most people would simply continue using proprietary software.

I also know that selling free software is allowed, but that is not a restriction. You can sell copies of free software, but you cannot restrict further distribution.

@noisytoot

Full agreement.

I'm hoping there are other mechanisms available.

@mrmasterkeyboard Is there a way to find all clones of your repos and submit them for removal too?

For clones that show up as GH clones because the original was on GH there's probably a way to list them all.

But what about when the original was not on GH and other people put their forks/working-copies there?

@dalias i genuinely think there is no option for this. i dunno. best hope is a DMCA maybe?

@mrmasterkeyboard The template looks like it lets you request removal of repos you don't own. But you'd first need a list of them.

@johnofrobotz

Giving back implies you took something at some point. I'm not under the impression that you ever took anything from HF.

@mrmasterkeyboard

@argv_minus_one @mrmasterkeyboard My intent was to give back to the dev community on GitHub - when it was pre-corp and more community driven.

Microsoft and Hugging Face profiting on public work is pretty brazen and a violation of public trust. But that's pretty much a day in the life of model training.

@mrmasterkeyboard this is so fucking stupid wtf how is this allowed unfortunately i feel like it's unlikely anything could be done from a legal perspective, given the license grants in the github terms of service;

"By making a repository public, you grant other Users a nonexclusive, worldwide license to use, display, perform and reproduce (by forking) Your Content through the Service as permitted by GitHub's functionality. You may grant additional rights by adopting a license. If you post Content you did not create or own, you are responsible for ensuring it is licensed under terms that permit these uses."

though, that's only on the copyright front. it seems likely that a case could be made with GDPR or similar regulation

oh... that's actually fucking disgusting

@mrmasterkeyboard Is it really that surprising? we made those repos public, gave a license, and basically anyone (yes even "companies") can read our code, learn from it, write their own code "better" because of what they read in our repo.
And companies sell their products...

@mrmasterkeyboard I'm almost certain that the UK grandfathered the GDPR into UK law on Brexit, but I'm not sure what might have happened since. But yes, what's described by that quote earlier in the thread would definitely be in violation of the GDPR! An opt-out after the fact is not enough.

IANAL, but I have an interest in this kind of stuff.

@sodiboo

@danielkasaj @mrmasterkeyboard
https://malus.sh/blog.html

@mrmasterkeyboard It’s also case-sensitive 🫠 And provides no feedback that it’s doing anything when you hit “check” (ie. Downloading the list) 🫠🫠

Thanks for the heads up

@mrmasterkeyboard

"We want to give developers agency over their source code by letting them decide whether or not it should be used to develop and evaluate machine learning models."

Fuck off assholes. If you wanted to give developers agency this would be opt-in, but we all know nobody actually likes this planet-destroying capital concentration machine so instead you're cosplaying as consent-respecting.