Okay, the world sucks and is pretty darn broken. But, I’ve got great news for you. You get to be the one who fixes it. You get to raise a single human who grows up to be put in charge of everything.
Your child grows up like a normal kid, but when they hit adolescence, they continue to get smarter and smarter. So smart, they make Einstein seem like a chump. Through it all, they keep most of their base beliefs about the world. (Unless they disprove those beliefs so thoroughly they must find some other path.) Whatever your child believes is “good” in the world, they carry that forward and push for it. Whatever you initially taught them: that is what they will strive for in the world.
They go to college, with a tearful goodbye, and you make them promise to video chat once a week. Every week they’re learning so much, you’re astonished. Within the first year, they somehow fund the right research to figure out how to make themselves immortal. Then they make themselves impervious to assassination. Your child invents a cryptocurrency system that works more efficiently than anyone thought was possible, and it solves a bunch of other coordination problems at scale in one fell swoop.
Now, when they call you once a week, it’s clear they are doing it out of habit, and out of respect for you, but they are so far advanced that they can’t really explain to little ol’ Mom and Dad what the heck they are actually doing. Now they have the resources to really push for their goals. Whatever your son or daughter thinks is good? Now that is what happens.
They continue to learn and grow, and the rest of us can no longer understand what it is like to be them. Their opinions of humanity change, due to their new perspective.
If you are a liberal leaning person, you might balk at perhaps forcing everyone under the thumb of a single person’s intellect. But maybe your child could make the world much better than it is?
If you are a conservative or libertarian, maybe you see this as your chance to rear a really smart king. They will make the world better for everyone, somehow.
Still. If you put your mind to it, you can imagine many ways in which this goes horrifically wrong. Your offspring’s viewpoint, whether you can put them in a right or left or conservative or liberal bucket, will be pushed to extremes, causing unforeseen side effects.
They’re so damn smart. You raised your child to think radically about the world, and empathize with all humanity. It becomes obvious to the world at large, as we implement more of their great ideas, that we should just put your offspring directly in charge of more and more things. They become a liberal, radical leader who creates a one world government in the span of less than a year. Borders are eradicated. Hunger is eliminated. Everyone is healthy. Miracles for life extension are shared. But then we find that our great leader somehow turned off our ability to procreate (because overpopulation would increase suffering, they patiently explain.) Instead of humanity expanding outward into the universe, we all sit at home, plugged into machines that make us happier than you can possibly imagine. You and the rest of us are all told by your offspring that this is for the best.
Is this good?
Or, maybe, it goes a bit differently.
They’re so damn smart. You raised your child to think very carefully about the world, and try not to rock the boat so hard it tips over. They become a conservative, reactionary leader who gradually grows into their role as the puppet-master of the global economy, keeping everything as it is. Like society is an exhibit under glass. Cancer and other killers are solved, but death is kept around, because it’s “necessary” to avoid overpopulation. Progress past a certain point is deliberately crushed by the dear leader, in order to keep things going in a guaranteed way. The world’s governments and economies are left in stasis. Any development of technology or art or political idea which is too “new” is silently destroyed in reaction, to keep us all from leaving the equilibrium. This is for the best, they patiently explain to you, every time you ask.
Is this good?
Did you raise them right?
Are you sure?
If you’ve ever raised a child, then it’s probably easier to imagine other ways this could go very, very wrong.
Now imagine it this way: instead of raising a normal child for the job, you’re raising a new strain of genetically enhanced human. They’re the first ever. A huge evo-devo-related gene therapy breakthrough has been made. Your new baby is predicted to grow up much, much faster than a child. In the first week, they pass through their toddler years. By the second week, they become a teenager. You begin to worry, because you can’t really control what they do. As the third week rolls around, it’s official: they are already smarter than you, and better at most things. Any time you try to punish or correct their behavior, they chuckle at you. And by the fourth week, they have already done the research to make themselves functionally immortal.
This new kind of person — let’s just agree to call them “King” — is not really going to be raised like a child. They won’t have the chance to grow up slowly, over time, where you can attempt parental course corrections. This King does not get taught carefully in a process full of making mistakes that you point out, and boundaries you set that keep them safe. Instead, they grow up in a hectic whirlwind of you trying to impress upon them what’s important, in a frantic race against time, before they spin off into the world.
How can you be sure they will learn what you are trying to impart?
How can you be sure the King’s decisions would be good for humanity? That they will stay good for humanity, over time?
In a month, as hard as it is to believe, your freakish child will be smarter than us all. They will be running everything.
Think how different you are from your parents. Think how similar you are, in other ways, the ways you would really rather not be.
What if they call you one day, their eyes filled with pity, and explain that they need to erase humanity and start over, because of fundamental biases in the way our cognition works? You beg your child, who is now more like a god than a king, for another path… but they stare through you at something you cannot see, say “Sorry, Dad. Sorry, Mom.” They hang up.
Are you worried about the value alignment problem yet? If not, why not?