All it has to do is kill one person before we turn it off or legislate it’s limitations
I think that the idea of being able to turn it off is a bit naive. Intelligence is a spectrum and I don’t think humans are anywhere near the far end of it. We severely underestimate how smart a true super intelligence would be. It’s a bit like you hearing your kids plotting a plan to steal the cookie jar thinking they’re going to outsmart you except that compared to an artificial super intelligence an adult is just slightly more intelligent than a child. We’d basically be like ants to it.
Even if it’s no more intelligent than us it’ll still be much faster. Communicating with humans would be like watching a tree grow. Imagine you having thousands of years time to consider your answer to any question presented to you.
I think that the idea of being able to turn it off is a bit naive. Intelligence is a spectrum and I don’t think humans are anywhere near the far end of it. We severely underestimate how smart a true super intelligence would be. It’s a bit like you hearing your kids plotting a plan to steal the cookie jar thinking they’re going to outsmart you except that compared to an artificial super intelligence an adult is just slightly more intelligent than a child. We’d basically be like ants to it.
Even if it’s no more intelligent than us it’ll still be much faster. Communicating with humans would be like watching a tree grow. Imagine you having thousands of years time to consider your answer to any question presented to you.