Chatgpt is going to get trained on thinking those two questions are duplicates and end up giving bullshit outdated answers to every question.

Stack overflow was the pioneer of hallucinations.

This is an ironic ChatGPT answer, meant to (rightfully) creep you out.

Looks like a chat bot instructed to say something contrarian

Hopefully a troll account after looking at other comments but who knows anymore

I took it as a joke because they can just change the rules whenever they want but Idk I might have misunderstood.

Looks like an AI crafted response to me.

Nope, it’s the establishment is cool, elon rocks type.

Nah, but the user is. Their post history is… interesting.

Jokes are supposed to be funny.

Poe’s law

That’s an important point, and and it ties into the way ChatGPT and other LLMs take advantage of a flaw in the human brain:

Because it impersonates a human, people are more inherently willing to trust it. To think it’s “smart”. It’s dangerous how people who don’t know any better (and many people that do know better) will defer to it, consciously or unconsciously, as an authority and never second guess it.

And the fact it’s a one on one conversation, no comment sections, no one else looking at the responses to call them out as bullshit, the user just won’t second guess it.

Which is why I used the former as an example and not the latter.

I’m not trying to make a general case for AI generated code here… just poking fun at the notion that a few errors will put people off using it.

The issue on the copyright front is the same kind of professional standards and professional ethics that should stop you from just outright copying open-source code into your application. It may be very small portions of code, and you may never get caught, but you simply don’t do that. If you wouldn’t steal a function from a copyleft open-source project, you wouldn’t use that function when copilot suggests it. Idk if copilot has added license tracing yet (been a while since I used it), but absent that feature you are entirely blind to the extent which it’s output is infringing on licenses. That’s huge legal liability to your employer, and an ethical coinflip.

Regarding understanding of code, you’re right. You have to own what you submit into the codebase.

The drawback/risks of using LLMs or copilot are more to do with the fact it generates the likely code, which means it’s statistically biased to generate whatever common and unnoticeable bugged logic exists in the average github repo it trained on. It will at some point give you code you read and say “yep, looks right to me” and then actually has a subtle buffer overflow issue, or actually fails in an edge case, because in a way that is just unnoticeable enough.

And you can make the argument that it’s your responsibility to find that (it is). But I’ve seen some examples thrown around on twitter of just slightly bugged loops; I’ve seen examples of it replicated known vulnerabilities; and we have that package name fiasco in the that first article above.

If I ask myself would I definitely have caught that? the answer is only a maybe. If it replicates a vulnerability that existed in open-source code for years before it was noticed, do you really trust yourself to identify that the moment copilot suggests it to you?

I guess it all depends on stakes too. If you’re generating buggy JavaScript who cares.

confirm the generated one worked. I think I’ll pass on tha

LLM wasn’t the right tool for the job, so search engine companies made their search engines suck so bad that it was an acceptable replacement.

Yeah, every time someone says how useful they find LLM for code I just assume they are doing the most basic shit (so far it’s been true).

That’s a 50% time reduction for the same output which sounds great to me.

I’d much rather let an LLM do the menial shit with my validation while I focus on larger problems such as system and API design, or creating rollback plans for major upgrades instead of expending mental energy writing something that has been written a thousand times. They’re not gonna rewrite your entire codebase, but they’re incredibly useful for the small stuff.

I’m not even particularly into LLMs, and they’re definitely not gonna change the world in the way big tech would like you to believe. However, to deny their usefulness is silly.

First a nearly unprecedented world-wide pandemic followed almost immediately by record-breaking layoffs then AI taking over the world, man it is really not a good time to start out as a newer developer. I feel so fortunate that I started working full-time as a developer nearly a decade ago.

Where did I say “Fuck 'em, all in on this stupid LLM bullshit!”?

But yes, there is a massive labor issue coming. That is why I am such a proponent of Universal Basic Income because there are not going to be enough jobs out there.

But as for training up the interns: Back in the day, do you know what “interns” did? And by “interns” I mean women because sexism but roll with me. Printing out and sorting punch cards. Compilers and general technical advances got rid of those jobs and pushed up where the “charlie work” goes.

These days? There are good internships/junior positions and bad ones. A good one actually teaches skills and encourages the worker to contribute. A bad one has them do the mindless grunt work that nobody else wants to. LLMs get rid of the latter.

And… I actually think that is good for the overall health of workers, if not the number (again, UBI). Because if someone can’t be trusted to write meaningful code without copying it off the internet and not even updating variable names? I don’t want to work with them. I spend too much of my workday babysitting those morons who are just here there to get some work experience so they can con their way into a different role and be someone else’s problem.

And experience will be gained the way it is increasingly being gained. Working on (generally open source) projects and interviewing for competitive internships where the idea is to take relatively low cost workers and have them work on a low ROI task that is actually interesting. It is better for the intern because they learn actual development and collaboration skills. And it is better for the staff because it is a way to let people work on the stuff they actually want to do without the massive investment of a few hundred hours of a Senior Engineer’s time.

And… there will be a lot fewer of those roles. Just like there were a lot fewer roles for artists as animation tools stopped requiring every single cell of animation to be hand drawn. And that is why we need to decouple life from work through UBI.

But also? If we have less internships that consist of “okay. good job. thanks for that. Next time can you at least try and compile your code? or pay attention to the squiggly red lines in your IDE? or listen to the person telling you that is wrong?”? Then we have better workers and better junior developers who can actually do more meaningful work. And we’ll actually need to update the interviewing system to not just be “did you memorize this book of questions from Amazon?” and we’ll have fewer “hot hires” who surprise everyone by being able to breath unassisted but have a very high salary because they worked for facebook.

Because, and here is the thing: LLMs are already as good, if not better than, an intern or junior engineer. And the companies that spend money on training up interns aren’t going to be rewarded. Under capitalism, there is no reason to “take one for the team” so that your competition can benefit.

Actually, nvidia recently announced RAG (Retrieval-Augmented Generation). Basically the idea is that you take an “off the shelf” LLM and then feed your local instance sensitive corporate data. It can then use that information in its responses.

So you really are “teaching” it every time you do a code review of the AI’s merge request and say “Well… that function doesn’t exist” or “you didn’t use useful variable names” and so forth. Which… is a lot more than I can say about a lot of even senior or principle engineers I have worked with over the years who are very much making mistakes that would get an intern assigned to sorting crayons.

Which, again, gets back to the idea of having less busywork. Less grunt work. Less charlie work. Instead, focus on developers who can actually contribute to a team and design meetings.

And the model I learned early in my career that I bring to every firm is to have interns be a reward for talented engineers and not a punishment for people who weren’t paying attention in Nose Goes. Teaching a kid to write a bunch of utility functions does nothing they didn’t learn (or not learn) in undergrad but it is a necessary evil… that an AI can do.

Instead, the people who are good at their jobs and contributing to the overall product? They probably have ideas they want to work on but don’t have the cycles to flesh out. That is where interns come into play. They work with those devs and other staff and learn what it means to actually be part of a team. They get to work on really cool projects and their mentors get to ALSO work on really cool projects but maybe focus more on the REALLY interesting parts and less on the specific implementation.

And result is that your interns are now actually developers who are worth a damn.

Also: One of the most important things to teach a kid is that they owe the company nothing. If they aren’t getting the raise they feel they deserve then they need to be updating their linkedin and interviewing elsewhere. That is good for the worker. And that also means that the companies that spend a lot of money training up grunts? They will lose them to the companies who are desperate for people who can lead projects and contribute to designs but haven’t been wasting money on writing unit tests.

Isn’t it amazing that places like this built on user support and contribution turn around and pull a “we don’t need you”?

In Europe GDPR gives you the right to have all your data deleted. All you do is send in a request and SO has to remove everything of yours, not just anonymize it. There are some exceptions for legal reasons, eg where financial transactions are involved, but comments should not be exempt.

deleted by creator

They don’t need to republish the 60 million questions, they just have to credit the authors, which are surely way fewer (but IANAL)

Ethically and logically it seems like output based on training data is clearly derivative work. Legally I suspect AI will continue to be the new powerful tool that enables corporations to shit on and exploit the works of countless people.

They are not. A derivative would be a translation, or theater play, nowadays, a game, or movie. Even stuff set in the same universe.

Expanding the meaning of “derivative” so massively would mean that pretty much any piece of code ever written is a derivative of technical documentation and even textbooks.

So far, judges simply throw out these theories, without even debating them in court. Society would have to move a lot further to the right, still, before these ideas become realistic.

I’m slow.