reminder that the whole "paperclip optimization" problem (the idea that an AI could/would destroy humanity if you told it to optimize for something because it would become the most optimal path) is actually... already there. they're called companies. it's all called capitalism. it's destroying the environment and exploiting humanity to optimize for money for shareholders.

@wxcafe it only makes sense if your AI is given shitty parameters.

This shit was solved like 80 years ago just give your AI's rules like "don't kill the humans" it's easy just adding another variable

@zardoz @wxcafe define human. What if a human must be killed to save more?

Asimov explored this a ton in i robot and foundation. The laws of robotics don't work

@enki @wxcafe I'm not saying that ethics is solved, just that you can avoid runaway optimization with a few simple rules. The reason why corporations behave so unethically is that they have little to no ethical rules, they are designed to optimize solely for profit. Adding those rules fixes far more problems than it creates.

@zardoz @enki @wxcafe I think that's incredibly naïve, sorry to say.
Runaway optimization is a actually pretty good parable for runaway capitalism.

Try defining "human" in axiomatic terms. You can't. An AI will not be guaranteed to interpret the rules in a way that is contiguous with actual human well-being. You need actual humans being in control for that.

Similar with capitalism. A corporation will always see benefit in dumping on the externalities.

We have to make it unprofitable, by hand.

@ansugeisler @enki @wxcafe Generally you don't do things like that axiomatically, that approach was abandoned because of how difficult it is. I don't think it's impossible to define "human" axiomatically but it is obviously difficult.

An AI can be guaranteed to follow the rules it has, the hard part is in selecting the rules so that they make the AI do things we like. Ethics is not easy but the hard part is not in programming in the ethics, the hard part is in figuring out what ethics to program in.

@ansugeisler @enki @wxcafe for instance, a paperclip-making AI should never have a reason to kill a human. Therefore you can add a rule: "never kill these things called humans, here's a million examples and a million non-examples of humans from a dataset with biases scrubbed, and here's the difference between alive and dead humans".

Basically an AI is a tool and you can give it safety parameters like any other tool. I don't know why this is controversial.

@zardoz @enki @wxcafe Where are you getting these absurd bits of nonsense from? What part of "Intelligence" do you think involves "working from an immutable set of unmodifiable rules"?

The ENTIRE problem is that an AI is absolutely certain to bend the rules it starts out with, because THAT IS WHAT LEARNING IS.

@ansugeisler @zardoz @enki the problem is actually that nobody cares about AIs, because AIs don't exist and probably never will. we're talking about capitalism here.

@ansugeisler @zardoz @enki the paperclip maximizer problem is a theoretical problem. saying "well, it's easy to solve it, just do [such and such]!" is the same as actually solving how santa will enter your home if your chimney is too narrow. nobody cares, the solution isn't the important part, the question is. the answer is "santa isn't real"

@ansugeisler @zardoz @enki here, AI isn't real. what is real is capitalism, and capitalism is exploiting people and trashing the planet. and the point of the mind experiment is to realize we need to stop it.

@wxcafe @ansugeisler @enki AI is real though, and it's mostly being used by said corporations to do heinous things.

The analogous solution to "put in some ethical rules to the AI" is to put in some ethical rules to our social/economic systems. Change the motivating forces behind economic decisions(like running a factory). Not as easy, but something we have to do.

I really thought this conversation was more about the AI bits than the critique-of-capitalism bits though, sorry for that.

@zardoz @wxcafe @ansugeisler @enki I think you're kinda missing the point of Paperclip Optimizer argument for AI; it's not that the *exact* result that matters, it's the difficulty of implementing a set of rules that would prevent AIs from acting in unexpected ways that would ultimately be harmful to humans. Say you told the AI "Thou shalt not kill" and "Make us happy"; so the AI decides to forcefully pump heroin into everyone. Then you say "No not like that", and then it forcefully insert electrodes into every brain to stimulate the pleasure center. It's about how difficult it is to pin down human morality with a set of rules.

@DissidentKitty @wxcafe @ansugeisler @enki yes, it's an ethics problem. Basically a different version of things like the trolley problem or the utility monster argument. But I'm trying to say that, if you do decide on some ethical system and build it I to an AI, it won't try its best to ignore the ethical system, it will always do its best to honor the rules and goals you give it, because that's how computers work.

@zardoz @enki @ansugeisler @wxcafe @DissidentKitty

i would say corporations are a form of decentralized intelligence. perhaps not independent enough of humans to be considered an AI. parasitic intelligence perhaps. one of the really fascinating things about corporations is how drastically they distort human behavior. the organization is composed of humans, many of which may believe they are doing good things, but as a whole do completely inhuman things. in many cases doing your job is completely inhuman but people do it anyway because they "have to".

like it isn't quite under the control of the board or stake/shareholders so you get this paperclip maximizing effect. I theorize that cooperatives can reign in this uncontrolled behavior, but even a co-op gives you plenty of tools to dissociate inhuman behavior from yourself in the name of whatever memetic entity the org ends up captured by.

@zardoz @wxcafe @enki No, this discussion is about the actually well-known properties of AI, as an entirely separate problem to Capitalism or Corporatism.

Doesn't matter who is using AI for which purposes, if they don't do a LOT of thinking about how ML actually works, the outcomes will be very bad.

It doesn't seem like you really know anything about ML.

@wxcafe @ansugeisler @zardoz @enki It’s so consistent how this always works.

1) Someone criticizes capitalism.
2) A reactionary has a clever comeback which involves total fiction — either a conspiracy theory, some kind of techno-utopianism, or just literal fiction.

@zardoz @wxcafe @ansugeisler @enki That’s weird because you’re saying that AI, which does not exist, can resolve the contradictions of capitalism.

@tesseract @wxcafe @ansugeisler @enki I have made no such claims, you can believe what you want about me though I suppose.

@ansugeisler @enki @wxcafe from how AI's are built both in theory and practice. They are built with some purpose, usually to optimize some function. The AI doesn't get to change this function, because if it did it would no longer be fulfilling its purpose and thus be a shitty AI. It is possible to choose the function such that killing humans is never the optimum.

@zardoz @wxcafe The problem with capitalists is that they write the rules

@zardoz @wxcafe “Well, I know I can’t kill them, but they keep getting in the way of my optimal solution. If I chop off all their limbs and hook them up to IVs, I can probably stack em pretty efficiently in that corner over there”

@vector @wxcafe this is why you make "treat the humans good" _PART OF_ the optimal solution. Optimizing for ethics.

@wxcafe i think it would be good if all matter in the solar system was transformed into paper clips i dont see anything bad about this

@wxcafe universal paperclips was about capitalism already

@wxcafe i highly recommend the actual play podcast "friends at the table" and particularly their cyberpunk season "counter/WEIGHT". no particular reason ;)

@wxcafe shareholders *and customers*. Where do you get your food? Who builds your computers and provides you with Internet?

Also, the nice thing about amoral processes is that you can get them to work for you by changing the rules of the game: carbon tax, UBI, and so on. (Easier said than done, though.)

@skybrian I'm too tired to explain that capitalism isn't necessary to have an economy, and that an economy isn't even necessary for production of goods and services, or that regulated capitalism isn't possible, sorry

@wxcafe too bad, sometime I'd like to find someone who can go in depth about some alternative systems.

Like, how much is different? Do alternative systems still have money and prices? And so on.

@wxcafe cool, happy to hear any reading suggestions you might have. Anything you particularly like?

@skybrian @wxcafe Marx's *Capital* for Communism and Kroptkin's *Conquest of Bread* for Anarcho-Communism are the basics.

@skybrian @wxcafe Capitalism is a supposed meritocracy where everyone is assigned a worth based on the free market. That concept alone is unethical, because it unfairly penalize people with conditions that prevents them from doing as much labor as others, or need special care to function as well. But even the meritocracy is a lie: people who already have money and power decide market value, so even if your skill is unique or you put in a lot of labor, they can still not pay you what should be yours in an actual meritocracy.

@skybrian @wxcafe In an Anarchist society, people perform labor based on their interests and talents, and everyone takes turn performing unwanted but necessary task that can't be automated. The product of your labor, such as artwork or invention, can be bartered with other for their product or services, without the middle man of money. Instead of competing to have jeweled trinkets at the cost of staving children, everyone takes what they need and leave the rest to others. This is known as Anacho-Communism, or Libertarian Socialism, or Stateless Marxism, and a host of other names.

@skybrian @wxcafe Some sort of Anarchism or Communism has actually been the system used by many societies in history. The reason they felled out of fashion was because of the rampant Imperialism and Colonialism in the past few centuries, where massive amount of labor and resources are required to plunder and enslave other civilizations, and Capitalism provides a way for that kind of centralized power without the culprit looking overtly dictatorial, and centuries of propaganda had made people forget these ways of lives even existed.

@skybrian @wxcafe Now more than ever, technology can enable everything from direct democracy on a massive scale to worldwide resource distribution, but instead we have social media pushing propaganda and robber barons holding all the wealth and power, because people had put power in their hands for their false promise of countless riches, and now people don't even remember what kind of power we used to have in the first place, what kind of world we can make if we'd only try.

@DissidentKitty @wxcafe So okay, let's say we don't have money or stores? How do computer chips get manufactured and where do you go to get a computer? Who builds the subway and who runs the sanitation department? How do you make sure there are enough bus drivers and mechanics? How do you decide where to put cell phone towers? How about airplanes and airports? Who builds and launches the GPS satellites?

The supply chains that build all our stuff and organizations that run all our services are mindbogglingly complicated and I'm not sure traditional society methods would work anymore?

Bu I only have the top-level summary and don't know much about alternatives to capitalism. Maybe someone has gone into more depth? Is there some modern introduction rather than going back to nineteenth century foundational texts from before most of the stuff we have now was invented?

@skybrian @wxcafe Capitalism and money didn't build your computers. People and their labor did. As long as there's demand, there's an incentive for labor, as you use your labor to satisfy demand for each other. And a lot of people do things because they like it - how many artists were starving before critics arbitrarily decide they're a maestro? How many techies work out of their garage before some venture arbitrarily throw money at them?

@skybrian @wxcafe Neither centralization nor monastery reward are prerequisite for organization. Spontaneous organization occurs in the smallest of organisms. How diverse would our media be if Disney doesn't own everything and queer artists aren't undervalued? How healthy would the Internet be if Google doesn't have a monopoly and coders aren't forced to maintain shitty code?

@DissidentKitty @wxcafe yeah, I agree that it's all people in the end, but it's not just a crowd, it's people who work together. Some of them know each other and maybe work in teams. Other people don't know each other but they sometimes need to briefly connect and work together anyway to solve problems that require coordination between people in faraway places.

As a garage tinkerer I don't care about venture capital but I need to buy tools and parts. I do lots of research to figure out what parts to get. I wouldn't get anywhere without trips to Home Depot and ordering electronic parts online. I'm not making my own chips and wire and diodes. Under this alternative system, would I have to barter for all that stuff or is there some storehouse I go to? If they don't know me, how do they decide whether to let me take stuff home? Maybe they have rules?

Also if the parts aren't available locally, do we have shipping? Who delivers all the stuff where it needs to go, and what do I give them for delivering something to me?

I'm not saying this can't be solved. Maybe someone has some of it figured out! But it seems like there are lots of problems to solve, and we need some kind of alternative framework for solving them if it's not the one based on money and prices that we're all familiar with.

In the past the breadbaker didnt mill their own flour, he got that from the miller. Not by barter or trade, but because the miller knew there was a need for flour in the community. You work together on goals and needs because you either share or support them.
@DissidentKitty @wxcafe

@wmd @DissidentKitty @wxcafe Not sure about the history but it sounds like free labor is expected, and on a vast scale too? I thought that was bad? It's okay for open source or Wikipedia, but I kinda do think most people should be paid for their labor, and not "pay what you want" either, that's a recipe for exploitation.

It reminds me of that story about Lolas in the Philippines. Room and board and being dependent on (not really) family isn't freedom and being completely dependent on a local community doesn't seem any more secure.

@skybrian @DissidentKitty @wxcafe

Ursula K Leguin played out an interesting example in her book 'the dispossessed' of a potential operation, based a 70's perspective. But what we need to watch out for is the application of capitalist para-dimes and symptoms. There is how the system is affecting/conditioning/influencing people's mindset now and what humans are capable of. The latter is a massive scope of possibilities, the former limits us more and more into one mode.

@skybrian @DissidentKitty @wxcafe

Petr Kropotkin wrote an interesting bit on that last bit btw:

We can only get there by envisioning what we are capable of and working towards that, not by focussing on the status quo.

@wmd @DissidentKitty @wxcafe It's been a long time since I read it, but yes, that was a good novel. Ursula K. Le Guin did a great job imagining what the experience of such a society might be like.

Still, there are lots of shortcuts you can take when making a society plausible enough to tell a good story about it.

@skybrian @DissidentKitty @wxcafe

yes, but it's in that a very helpful book in how we can imagine a system to work. If we just use conditioning by capitalism as 'human nature', we're just accepting the status quo.

We need to imagine the possible -outside- of our constraints and find a path to there.

@wmd @DissidentKitty @wxcafe I think that might be a promising way to think about it. There doesn't seem to be a complete do-everything system waiting in the wings that's ready to take over everything from capitalism? Lots of pieces, though. Examples from open source, Wikipedia. Flawed but interesting examples from the past. Possibly they might be extended?

Coming up with each new framework for collaboration, some kind of "free software but for X" would be a lot of work and a major victory, just like free software/open source was. To make it work and popularize it is a design problem requiring imagination, experimentation, and cultural innovation.

And it's not going to be pure, just like open source isn't pure. It's gotta work without being isolated from capitalism. Boundaries are important.

@skybrian @DissidentKitty @wxcafe And there is a lot of inspiring work out there. From recent books like 'walk away' to 20k year old cultures.

Sign in to participate in the conversation

This is a mastodon instance for social justice activists, LGBTQIA+ people, and activists in general See the Goals and technical details, and Rules and privacy policy pages for more information