Noah's take was an instant unsubscribe for me. I don't mind when smart people poke around at the edges of their expertise - Lord knows I do it. But that felt wildly arrogant and showcased a lack of humility that I find untrustworthy.
Yes, but according to Dario Amodei it “will actually be possible to cure most human disease in ~5-10 years” Given the biological knowledge gain implied by this claim, maybe Noah is onto something. Maybe if this claim and others like it turn out to be wrong, then we’ll still be discussing hypotheticals 10 years from now. I expect to be reading hypotheticals (and AI post-mortems), 10 years from now.
One scenario (which I find implausible) that Yudkowsky describes for AI caused human extinction is the AI creates self-replicating nanobots which get inside and kill everyone on cue. Now, my impression is that we are all under attack by self-replicating nanobots all the time and have been for however many hundreds of millions of years. What are your thoughts on such a scenario?
Setting aside the issue of AI-engineered pathogens for a second, I'm currently reading Annie Jacobsen's "Biological War: A Scenario", and I'm wondering how plausible you find that. The Russians apparently have a biological warfare program carried over from the Soviet era, which has weaponized pathogens like smallpox, Marburg, and plague. In her fictional scenario, a Russian scientist feeling guilty tries to destroy the his research and inadvertently ends up freeing pneumonic plague which has been genetically modified to induce euphoria in the early stages to keep the patient active and spreading even when they begin to get symptomatic. This is supposed to be well-researched and based on factual information from defecting Russian scientists. In the scenario, this basically leads to complete society breakdown and mass death. Is this plausible to you?
If the definition of supervirus is “kills 90% of the human population” (or even just 10%) then no, measles won’t do that. A world-wide measles outbreak would be bad, but it wouldn’t end civilization.
I tend to cringe when a scientist says it is impossible to "practically" make or do such and such. Here I don't include those things that are physically impossible (like, accelerating to the velocity of light, or perpetual machines with no energy input). Time and time again human ingenuity had falsified proclamations of limits to practical attainment.
There is no a priori reason to assume there can be no human virus with 90% lethality. True, there is no such *known* human virus: it could be unknown because such a virus might be rather unfit to propagate through to a pandemic status or because of human genetic diversity. But there are bird flu types (H5N1, H5N8) that are 90-100% lethal to (admittedly genetically nearly homogeneous) poultry populations. There is zero assurance that a virus cannot be engineered, which uses a highly invariant human cell surface protein as a receptor, such as the Na-K ATPase transporter subunit or the primary glucose transporter. Whether a high-school student can make it? I don't know but I never underestimate the younger generations.
Not just 90% lethality. 90% lethality while spreading easily and spreading primarily during the asymptomatic period, so nobody notices they are infected until it's too late. I think such a scenario is extremely unlikely. Maybe not impossible, but sufficiently unlikely that I'd rather worry about other things.
And, in any case, the only thing we can do to protect ourselves from such a virus is to improve public health and invest in rapid vaccine development.
I think Noah’s difficulty is, like most writers, he cannot understand that some things are infinitely more complex than language.
Noah's take was an instant unsubscribe for me. I don't mind when smart people poke around at the edges of their expertise - Lord knows I do it. But that felt wildly arrogant and showcased a lack of humility that I find untrustworthy.
Yes, but according to Dario Amodei it “will actually be possible to cure most human disease in ~5-10 years” Given the biological knowledge gain implied by this claim, maybe Noah is onto something. Maybe if this claim and others like it turn out to be wrong, then we’ll still be discussing hypotheticals 10 years from now. I expect to be reading hypotheticals (and AI post-mortems), 10 years from now.
One scenario (which I find implausible) that Yudkowsky describes for AI caused human extinction is the AI creates self-replicating nanobots which get inside and kill everyone on cue. Now, my impression is that we are all under attack by self-replicating nanobots all the time and have been for however many hundreds of millions of years. What are your thoughts on such a scenario?
Setting aside the issue of AI-engineered pathogens for a second, I'm currently reading Annie Jacobsen's "Biological War: A Scenario", and I'm wondering how plausible you find that. The Russians apparently have a biological warfare program carried over from the Soviet era, which has weaponized pathogens like smallpox, Marburg, and plague. In her fictional scenario, a Russian scientist feeling guilty tries to destroy the his research and inadvertently ends up freeing pneumonic plague which has been genetically modified to induce euphoria in the early stages to keep the patient active and spreading even when they begin to get symptomatic. This is supposed to be well-researched and based on factual information from defecting Russian scientists. In the scenario, this basically leads to complete society breakdown and mass death. Is this plausible to you?
You seem to suggest that tinkering with the measles virus would be a viable route (to a devastating supervirus), rather than AI wizardry.
If the definition of supervirus is “kills 90% of the human population” (or even just 10%) then no, measles won’t do that. A world-wide measles outbreak would be bad, but it wouldn’t end civilization.
I tend to cringe when a scientist says it is impossible to "practically" make or do such and such. Here I don't include those things that are physically impossible (like, accelerating to the velocity of light, or perpetual machines with no energy input). Time and time again human ingenuity had falsified proclamations of limits to practical attainment.
There is no a priori reason to assume there can be no human virus with 90% lethality. True, there is no such *known* human virus: it could be unknown because such a virus might be rather unfit to propagate through to a pandemic status or because of human genetic diversity. But there are bird flu types (H5N1, H5N8) that are 90-100% lethal to (admittedly genetically nearly homogeneous) poultry populations. There is zero assurance that a virus cannot be engineered, which uses a highly invariant human cell surface protein as a receptor, such as the Na-K ATPase transporter subunit or the primary glucose transporter. Whether a high-school student can make it? I don't know but I never underestimate the younger generations.
Not just 90% lethality. 90% lethality while spreading easily and spreading primarily during the asymptomatic period, so nobody notices they are infected until it's too late. I think such a scenario is extremely unlikely. Maybe not impossible, but sufficiently unlikely that I'd rather worry about other things.
And, in any case, the only thing we can do to protect ourselves from such a virus is to improve public health and invest in rapid vaccine development.