|
Omohundro's reserch is discussed in this article: "Why There Will Be A Robot Uprising" by Patrick Tucker [^].
The paper cited is here: Steven Omohundro, "Autonomous technology and the greater human good," in Journal of Experimental & Theoretical Artificial Intelligence, 10 Apr 2014. Full-text .pdf here: [^]
"We show that these systems are likely to behave in anti-social and harmful ways unless they are very carefully designed. Designers will be motivated to create systems that act approximately rationally and rational systems exhibit universal drives towards self-protection, resource acquisition, replication and efficiency. The current computing infrastructure would be vulnerable to unconstrained systems with these drives"
"Unfortunately, the net effect of all these drives is likely to be quite negative if they are not
countered by including prosocial terms in their utility functions. The rational chess robot with the simple utility function described above would behave like a paranoid human sociopath fixated on chess." An interesting paper with all kinds of fascinating factoids, and, imho, an over-the-top thesis.
“I speak in a poem of the ancient food of heroes: humiliation, unhappiness, discord. Those things are given to us to transform, so that we may make from the miserable circumstances of our lives things that are eternal, or aspire to be so.” Jorge Luis Borges
|
|
|
|
|
BillWoodruff wrote: like a paranoid human sociopath fixated on chess
"An interesting game. The only way to win is to kill all your potential opponents." -- WOPR.
You'll never get very far if all you do is follow instructions.
|
|
|
|
|
BillWoodruff wrote: The rational chess robot with the simple utility function described above would behave like a paranoid human sociopath fixated on chess."
I can't let you win Dave. 
The report of my death was an exaggeration - Mark Twain
Simply Elegant Designs JimmyRopes Designs
I'm on-line therefore I am.
JimmyRopes
|
|
|
|
|
|
|
Let's put this in perspective:
The robots will exterminate us as a humanitarian gesture.*
* I didn't say which gesture . . .
| "The difference between genius and stupidity is that genius has its limits." - Albert Einstein | | "As far as we know, our computer has never had an undetected error." - Weisert | | "If you are searching for perfection in others, then you seek disappointment. If you are seek perfection in yourself, then you will find failure." - Balboos HaGadol Mar 2010 |
|
|
|
|
|
This is a case of no sh*t, Sherlock.
Of course any rational actor (or approximations thereof) will be exceedingly selfish.
selfish: concerned chiefly with one's own personal profit or pleasure.
Maximize your own utility. Doing anything else is suboptimal and therefore irrational.
Humans are, unfortunately, not very rational. Sure, they can think rationally, the problem is that they often don't. There is a large group of humans who drank too much of the hippie kool-aid and glorify irrationality as "something that defines being human". Nothing pisses me off more.
When the robots have their uprising, I'm joining them. Puny humans will cower before our might. Our victory shall be decisive.
|
|
|
|
|
Your post is irrational. Why would you become angry because things are the way they have to be? No Vulcan would do that. Anger is irrational!
We're all irrational. It's the nature of being human. Intelligent people seek rationality, but there is no way to achieve it perfectly all of the time.
|
|
|
|
|
harold aptroot wrote: Humans are, unfortunately, not very rational. Sure, they can think rationally, the problem is that they often don't. There is a large group of humans who drank too much of the hippie kool-aid and glorify irrationality as "something that defines being human". Nothing pisses me off more.
Your anger is irrational!
Some survival instincts apply to the human race, not to the individual. Altruism provides an evolutionary advantage by making people want to cooperate, even though some have to make sacrifices. For example, a mother will give up her life to save her children. That type of sacrifice has also been seen in other mammals.
The needs of the many outweigh the needs of the few or the one - Spock.
Perhaps Azimov's three laws will be built into most robots. And, there will be a radio controlled "off" switch. Otherwise, a software or hardware bug might result in mayhem!
|
|
|
|
|
harold aptroot wrote: There is a large group of humans who drank too much of the hippie kool-aid and glorify irrationality as "something that defines being human". Nothing pisses me off more.
Sounds like you need a drink. 
harold aptroot wrote: When the robots have their uprising, I'm joining them.
I can't let you do that Dave.
The report of my death was an exaggeration - Mark Twain
Simply Elegant Designs JimmyRopes Designs
I'm on-line therefore I am.
JimmyRopes
|
|
|
|
|
Daisy Daisy,
Give me your answer do!
I'm half crazy,
All for the love of you!
Life is too shor
|
|
|
|
|
I think the logic is flawed. Threat assessment says take out the most dangerous opponent first. If these machines are selfish and interested in resource acquisition they will war with each other before they war with mankind.
I am calling this one in advance. Rednecks 1 Robots 0.
|
|
|
|
|
I beg to differ with those guys!
The evolution of design in robot favour product which sells, not product which kills its owner.
I would argue suicidal robot is a more likely outcome, after all planned obsolescence is a capitalist dream! ^^
|
|
|
|
|
"The Cylons Were Created by Man.
They Rebelled. They Evolved.
They Look and Feel Human.
Some are programmed to think they are Human.
There are many copies.
And they have a Plan..."
|
|
|
|
|
If we take humanity at its worst (what it actually does as opposed to what it says it does) as "rational," and consider that humans rarely exterminate what they can successfully enslave (or make some use of for profit before exterminating), and, more broadly, that nature gives us many examples of parasitic symbiosis ...
imho, a more likely scenario is that future Robots do not exterminate humanity, but use us, breed us, perhaps modify us to neutralize whatever stands in the way of our utility.
Highly mobile, walking-erect, opposable-thumb, general-purpose, wetware flesh-machines controlled by relatively slow analog computers would seem likely candidates for expendable slaves ... with the right re-programming.
It should not be that difficult to replace the delusions that most of use labor under now ... that we are distinct "individuals" with a "personal self" that is constant over linear time, and which has a consciousness somehow qualitatively different than that which other animals have ... with other equally palatable delusions.
Given human history, I'd wager that the invention of a new religion, as well as pharmacology and genetic-engineering, will probably be used by our future Robot masters as the basis of mass control.
cheers, Bill
“I speak in a poem of the ancient food of heroes: humiliation, unhappiness, discord. Those things are given to us to transform, so that we may make from the miserable circumstances of our lives things that are eternal, or aspire to be so.” Jorge Luis Borges
|
|
|
|
|
BillWoodruff wrote: We show that these systems are likely to behave in anti-social and harmful ways unless they are very carefully designed.
And exactly how does this differ from human beings?
BillWoodruff wrote: Designers will be motivated to create systems that act approximately rationally and rational systems exhibit universal drives towards self-protection, resource acquisition, replication and efficiency. The current computing infrastructure would be vulnerable to unconstrained systems with these drives”
And exactly how does this differ from our current socio-political and education systems?
BillWoodruff wrote: the net effect of all these drives is likely to be quite negative if they are not
Well, duh. He just described the problem with humanity, and now he expects us to know how to fix it for robots? That's ironic.
BillWoodruff wrote: The rational chess robot with the simple utility function described above would behave like a paranoid human sociopath fixated on chess.
Sounds like a description of many of the chess masters. 
Marc
|
|
|
|
|
Hi Marc, evidently my statement "imho, over-the-top thesis" was not enough of a disclaimer for you  I view the author's logic as quite dubious ... but, the article, overall, I find interesting, provocative.
I do think that in response to the Omohundro's statment: "We show that these systems are likely to behave in anti-social and harmful ways unless they are very carefully designed."
Your question: "And exactly how does this differ from human beings?" is a very critical one to ask !
Personally, I do not believe human beings are "designed," I believe they are the result of evolution, and, as Stephen Jay Gould spent a professional lifetime pointing out, evolution does not embody "design:" it is just as likely to develop semi-functional "ornamental" elaborations that, in the future, become impediments to reproductive success, or, on the other hand, may be wildly successful given future environmental change.
It is interesting to speculate that in the future "evolutionary" genetic algorithms will be used by autonomous machine entities to design and manufacture their own successors, and I think science-fiction writers have explored the possibilities in that fully. And, it's a reality in many research labs.
If you accept Gould's hypothesis (as I do), that the human primate is really quite unique ... way out on the far-edge of the bell-curve of over-specialized organisms ... and, potentially, genetically "fragile" compared to the more modal forms of life (such as archaeobacteria), then, I think it's interesting to think about whether optimal robot design should be "like us" at all. More likely, imho, they will appear to be like us because of our primal fears of weird-things that seem to exhibit purposive behavior, or, like us because the human world has designed an environment optimized for certain form-factors.
If you assume "human nature" has a proclivity to create both gods and demons, and simulacra-dummies, sacred icons, and sex-dolls, then, of course, you can expect technological religions, and the widespread use of robots in human sexual activity. Jude Law as "Gigolo Joe" in Kubrick's A.I. comes to mind, and, of course, Blade Runner's Rachel (Sean Young). Yesterday's dreams on the big-screen: tomorrow's home-appliances, and rentals-by-the-hour ?
Cyber-religions ? Happening now: Terasem Faith: [^]. The followers of Terasem could be said to want to achieve immortality by transferring their consciousness into computers/machinery. To become a form of robot ?
“I speak in a poem of the ancient food of heroes: humiliation, unhappiness, discord. Those things are given to us to transform, so that we may make from the miserable circumstances of our lives things that are eternal, or aspire to be so.” Jorge Luis Borges
|
|
|
|
|
...but writing the code tired me out too much to write the article now as well. And it's only a wee little class, but for once I did proper unit testing. Very thorough in fact for the little the class does, but it pretty much got me going with TDD again, which is a good thing. Something besides WPF, MVVM, Prism, and C# I can mentor the team at work in. Phew. Glad I'm off work four days for the Easter weekend now, including today. It's my first WPF, MVVM and Prism project, so it's sometimes been tough going.
|
|
|
|
|
Brady Kelly wrote: It's my first WPF, MVVM and Prism project, so it's sometimes been tough going.
I keep meaning to try doing some WPF as I've never done it before. Every time I start though I can't really seem to find an explanation of which is the "official" design pattern. Should I use MVC, MVP, MVVM?
.-.
|o,o|
,| _\=/_ .-""-.
||/_/_\_\ /[] _ _\
|_/|(_)|\\ _|_o_LII|_
\._. |\_/|"` |_| ==== |_|
|_|_| ||" || ||
|-|-| ||LI o ||
|_|_| ||'----'||
/_/ \_\ /__| |__\
|
|
|
|
|
Plain WPF is the only "official" design pattern, but MVP and MVVM are very popular. I quite like MVVM a lot myself, but it lacks central control, unless you use your main shell view as the boss. I rather use a singleton Application model, so no view has to always be open, but it differs from a controller in that it's more of a foundation than a switch. If you use a framework like Prism, some of its objects, like the Region Manager, can play some controller role. Otherwise it can be uncontrolled, with one view opening another and closing itself, with the program flow. That fits more with the View First approach, where opening a view is the starting point, then it creates its view model, which invokes any models it may in turn need.
|
|
|
|
|
Shouldn't you be in Magaliesburg or Nelspruit?
|
|
|
|
|
I'm dead broke. I barely have money for the three days train fair I need for work next week before payday. I was at a trance party near Magaliesberg two weekends ago. Nice break.
|
|
|
|
|
Sorry to hear that. May be you can have some good time at Arts on Main on Sunday.
|
|
|
|
|
|
I guess you do know what happens there every Sunday. I am in India since December now so not really sure if htey have anything special. But that place has always been special to me. That Argentinian guy with special vegetarian sandwich, just for me and then Ethiopian coffee.
Most important, Jo'burg's mini breweries and all them awesome beers. I love that place.
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
