kottke.org posts about artificial intelligence
In this video, you can watch a simple neural network learn how to navigate a video game race track. The program doesn’t know how to turn at first, but the car that got the furthest in the first race (out of 650 competitors) is then used as the seed for the next generation. The winning cars from each generation are used to seed the next race until a few of them make it all the way around the track in just the 4th generation.
I think one of the reason I find neural network training so fascinating is that you can observe, in a very simple and understandable way, the basic method by which all life on Earth evolved the ability to do things like move, see, swim, digest food, echolocate, grasp objects, and use tools. (via dunstan)
NVIDIA has been doing lots of interesting things with deep learning algorithms lately (like AI-Generated Human Faces That Look Amazingly Real). Their most recent effort is the development and training of a program that takes rough sketches and converts them into realistic images.
A novice painter might set brush to canvas aiming to create a stunning sunset landscape — craggy, snow-covered peaks reflected in a glassy lake — only to end up with something that looks more like a multi-colored inkblot.
But a deep learning model developed by NVIDIA Research can do just the opposite: it turns rough doodles into photorealistic masterpieces with breathtaking ease. The tool leverages generative adversarial networks, or GANs, to convert segmentation maps into lifelike images.
Here’s a post I did 10 years ago that shows how far sketch-to-photo technology has come.
A machine learning algorithm programmed by Dr. Jae Ho Sohn can look at PET scans of human brains and spot indicators of Alzheimer’s disease with a high level of accuracy an average of 6 years before the patients would receive a final clinical diagnosis from a doctor.
To train the algorithm, Sohn fed it images from the Alzheimer’s Disease Neuroimaging Initiative (ADNI), a massive public dataset of PET scans from patients who were eventually diagnosed with either Alzheimer’s disease, mild cognitive impairment or no disorder. Eventually, the algorithm began to learn on its own which features are important for predicting the diagnosis of Alzheimer’s disease and which are not.
Once the algorithm was trained on 1,921 scans, the scientists tested it on two novel datasets to evaluate its performance. The first were 188 images that came from the same ADNI database but had not been presented to the algorithm yet. The second was an entirely novel set of scans from 40 patients who had presented to the UCSF Memory and Aging Center with possible cognitive impairment.
The algorithm performed with flying colors. It correctly identified 92 percent of patients who developed Alzheimer’s disease in the first test set and 98 percent in the second test set. What’s more, it made these correct predictions on average 75.8 months — a little more than six years — before the patient received their final diagnosis.
This is the stuff where AI is going to be totally useful…provided the programs aren’t cheating somehow.
The opening line of Madeline Miller’s Circe is: “When I was born, the name for what I was did not exist.” In Miller’s telling of the mythological story, Circe was the daughter of a Titan and a sea nymph (a lesser deity born of two Titans). Yes, she was an immortal deity but lacked the powers and bearing of a god or a nymph, making her seem unnervingly human. Not knowing what to make of her and for their own safety, the Titans and Olympic gods agreed to banish her forever to an island.
Here’s a photograph of a woman who could also claim “when I was born, the name for what I was did not exist”:
The previous line contains two lies: this is not a photograph and that’s not a real person. It’s an image generated by an AI program developed by researchers at NVIDIA capable of borrowing styles from two actual photographs of real people to produce an infinite number of fake but human-like & photograph-like images.
We propose an alternative generator architecture for generative adversarial networks, borrowing from style transfer literature. The new architecture leads to an automatically learned, unsupervised separation of high-level attributes (e.g., pose and identity when trained on human faces) and stochastic variation in the generated images (e.g., freckles, hair), and it enables intuitive, scale-specific control of the synthesis.
The video offers a good look at how this works, with realistic facial features that you can change with a slider, like adjusting the volume on your stereo.
Photographs that aren’t photographs and people that aren’t people, born of a self-learning machine developed by humans. We’ll want to trust these images because they look so real, especially once they start moving and talking. I wonder…will we soon seek to banish them for our own safety as the gods banished Circe?
Update: This Person Does Not Exist is a single serving site that provides a new portrait of a non-existent person with each reload.
The Lumière brothers were among the first filmmakers in history and from 1896 to 1900, they shot several scenes around Paris. Guy Jones remastered the Lumière’s Paris footage, stabilized it, slowed it down to a natural rate, and added some Foley sound effects. As Paris today looks very similar to how it did then, it’s easy to pick out many of the locations seen in this short compilation: the Tuileries, the Notre-Dame, Place de la Concorde, and of course the Eiffel Tower, which was completed only 8 years before filming. Here’s the full location listing:
0:08 - Notre-Dame Cathedral (1896)
0:58 - Alma Bridge (1900)
1:37 - Avenue des Champs-Élysées (1899)
2:33 - Place de la Concorde (1897)
3:24 - Passing of a fire brigade (1897)
3:58 - Tuileries Garden (1896)
4:48 - Moving walkway at the Paris Exposition (1900)
5:24 - The Eiffel Tower from the Rives de la Seine à Paris (1897)
See also A Bunch of Early Color Photos of Paris, Peter Jackson’s documentary film featuring remastered film footage from World War I, and lots more film that Jones has remastered and uploaded. (via open culture)
Update: Just as he did with the NYC footage from 1911, Denis Shiryaev has used machine learning algorithms to restore the Lumières film of Paris — it’s been upsampled to 4K & 60 fps, sharpened, and colorized.
Again, there are some obvious artifacts and the colorization is distracting, but the result is impressive for push-button. (via open culture)
This spreadsheet lists a number of ways in which AI agents “cheat” in order to accomplish tasks or get higher scores instead of doing what their human programmers actually want them to. A few examples from the list:
Neural nets evolved to classify edible and poisonous mushrooms took advantage of the data being presented in alternating order, and didn’t actually learn any features of the input images.
In an artificial life simulation where survival required energy but giving birth had no energy cost, one species evolved a sedentary lifestyle that consisted mostly of mating in order to produce new children which could be eaten (or used as mates to produce more edible children).
Agent kills itself at the end of level 1 to avoid losing in level 2.
AI trained to classify skin lesions as potentially cancerous learns that lesions photographed next to a ruler are more likely to be malignant.
That second item is a doozy! Philosopher Nick Bostrom has warned of the dangers of superintelligent agents that exploit human error in programming them, describing a possible future where an innocent paperclip-making machine destroys the universe.
The “paperclip maximiser” is a thought experiment proposed by Nick Bostrom, a philosopher at Oxford University. Imagine an artificial intelligence, he says, which decides to amass as many paperclips as possible. It devotes all its energy to acquiring paperclips, and to improving itself so that it can get paperclips in new ways, while resisting any attempt to divert it from this goal. Eventually it “starts transforming first all of Earth and then increasing portions of space into paperclip manufacturing facilities”.
But some of this is The Lebowski Theorem of machine superintelligence in action. These agents didn’t necessarily hack their reward functions but they did take a far easiest path to their goals, e.g. the Tetris playing bot that “paused the game indefinitely to avoid losing”.
Update: A program that trained on a set of aerial photographs was asked to generate a map and then an aerial reconstruction of a previously unseen photograph. The reconstruction matched the photograph a little too closely…and it turned out that the program was hiding information about the photo in the map (kind of like in Magic Eye puzzles).
We claim that CycleGAN is learning an encoding scheme in which it “hides” information about the aerial photograph x within the generated map Fx. This strategy is not as surprising as it seems at first glance, since it is impossible for a CycleGAN model to learn a perfect one-to-one correspondence between aerial photographs and maps, when a single map can correspond to a vast number of aerial photos, differing for example in rooftop color or tree location.
Add finding Waldo to the long list of things that machines can do better than humans. Creative agency Redpepper built a program that uses Google’s drag-and-drop machine learning service to find the eponymous character in the Where’s Waldo? series of books. After the AI finds a promising Waldo candidate, a robotic arm points to it on the page.
While only a prototype, the fastest There’s Waldo has pointed out a match has been 4.45 seconds which is better than most 5 year olds.
I know Skynet references are a little passé these days, but the plot of Terminator 2 is basically an intelligent machine playing Where’s Waldo I Want to Kill Him. We’re getting there!
Update: Prior art: Hey Waldo, HereIsWally, There’s Waldo!
NVIDIA trained a deep learning framework to take videos filmed at 30 fps and turn them into slow motion videos at the equivalent of 240 or even 480 fps. Even though the system is guessing on the content in the extra frames, the final results look amazingly sharp and lifelike.
“There are many memorable moments in your life that you might want to record with a camera in slow-motion because they are hard to see clearly with your eyes: the first time a baby walks, a difficult skateboard trick, a dog catching a ball,” the researchers wrote in the research paper. “While it is possible to take 240-frame-per-second videos with a cell phone, recording everything at high frame rates is impractical, as it requires large memories and is power-intensive for mobile devices,” the team explained.
With this new research, users can slow down their recordings after taking them.
Using this technique and what Peter Jackson’s team is doing with WWI footage, it would be interesting to clean up and slow down all sorts of archival footage (like the Zapruder film, just to choose one obvious example).
Using a JavaScript machine learning package called TensorFlow.js, Abhishek Singh built a program that learned how to translate sign language into verbal speech that an Amazon Alexa can understand. “If voice is the future of computing,” he signs, “what about those who cannot [hear and speak]?”
See also how AirPods + the new Live Listen feature “could revolutionize what it means to be hard of hearing”.
A research team at MIT’s Media Lab have built what they call the “world’s first psychopath AI”. Meet Norman…it provides creepy captions for Rorschach inkblots. A psychopathic AI is freaky enough, but the kicker is that they used Reddit as the dataset. That’s right, Reddit turns mild-mannered computer programs into psychopaths.
Norman is an AI that is trained to perform image captioning; a popular deep learning method of generating a textual description of an image. We trained Norman on image captions from an infamous subreddit (the name is redacted due to its graphic content) that is dedicated to document and observe the disturbing reality of death. Then, we compared Norman’s responses with a standard image captioning neural network (trained on MSCOCO dataset) on Rorschach inkblots; a test that is used to detect underlying thought disorders.
Here’s a comparison between Norman and a standard AI when looking at the inkblots:
Welp! (via @Matt_Muir)
Update: This project launched on April 1. While some of the site’s text at launch and the psychopath stuff was clearly a prank, Norman appears to be legit.
Cameras that can take usable photos in low light conditions are very useful but very expensive. A new paper presented at this year’s IEEE Conference on Computer Vision and Pattern Recognition shows that training an AI to do image processing on low-light photos taken with a normal camera can yield amazing results. Here’s an image taken with a Sony a7S II, a really good low-light camera, and then corrected in the traditional way:
The colors are off and there’s a ton of noise. Here’s the same image, corrected by the AI program:
Pretty good, right? The effective ISO on these images has to be 1,000,000 or more. A short video shows more of their results:
It would be great to see technology like this in smartphones in a year or two.
Hello, it is I, once and future Kottke.org guest editor Aaron Cohen. In the years since my objectively wonderful and technically perfect stints posting skateboarding and BMX videos here, I opened an ice cream shop in Somerville, MA called Gracie’s Ice Cream. As an ice cream professional and Kottke.org alumni, I’m not qualified for much except for writing about ice cream on Kottke.org (and posting skateboarding and BMX videos which I will do again some day). Now that I’ve mentioned Kottke.org 4 times in the first paragraph per company style guide, let’s get on with the post.
At aiweirdness.com, researcher Janelle Shane trains neural networks. And, reader, as an ice cream professional, I have a very basic understanding of what “trains neural networks” means [Carmody, get in here], but Shane recently shared some ice cream flavors she created using a small dataset of ice cream flavors infected with a dataset of metal bands, along with flavors created by an Austin middle school coding class. The flavors created by the coding class are not at all metal, but when it comes to ice cream flavors, this isn’t a bad thing. Shane then took the 1600 original flavor non-metal ice cream flavor dataset and created additional flavors.
The flavors are grouped together loosely based on much they work on ice cream flavors. I figured I’d pick a couple of the flavor names and back into the recipes as if I was on a Chopped-style show where ice cream professionals are given neural network-created ice cream flavor names and asked to produce fitting ice cream flavors. I have an asterisk next to flavors I’m desperate to make this summer.
From the original list of metal ice cream flavors:
*Silence Cherry - Chocolate ice cream base with shredded cherry.
Chocolate Sin - This is almost certainly a flavor name somewhere and it’s chocolate ice cream loaded with multiple formats of chocolate - cookies, chips, cake, fudge, you name it.
*Chocolate Chocolate Blood - Chocolate Beet Pie, but ice cream.
From the students’ list, some “sweet and fun” flavors:
Honey Vanilla Happy - Vanilla ice cream with a honey swirl, rainbow sprinkles.
Oh and Cinnamon - We make a cinnamon ginger snap flavor once in a while, and I’m crushed we didn’t call it “Oh and Cinnamon.” Probably my favorite, most Gracie’s-like flavor name of this entire exercise.
From the weirder list:
Chocolate Finger - Chocolate ice cream, entire Butterfinger candy bars like you get at the rich houses on Halloween.
Crackberry Pretzel - Salty black raspberry chip with chocolate covered pretzel.
Worrying and ambiguous:
Brown Crunch - Peanut butter Heath Bar.
Sticky Crumple - Caramel and pulverized crumpets.
Cookies and Green - Easy. Cookies and Cream with green dye.
“Trendy-sounding ice cream flavors”:
Lime Cardamom - Sounds like a sorbet, to be honest.
Potato Chocolate Roasted - Sweet potato ice cream with chocolate swirl.
Chocolate Chocolate Chocolate Chocolate Road - We make a chocolate ice cream with chocolate cookie dough called Chocolate Chocolate Chocolate Chip Cookie Dough, so this isn’t much of a stretch. Just add chocolate covered almonds and we’re there.
More metal ice cream names:
*Swirl of Hell - Sweet cream ice cream with fudge, caramel, and Magic Shell swirls.
Nightham Toffee - This flavor sounds impossibly British so the flavor is an Earl Gray base with toffee bits mixed in.
Yesterday, Google announced an AI product called Duplex, which is capable of having human-sounding conversations. Take a second to listen to the program calling two different real-world businesses to schedule appointments:1
More than a little unnerving, right? Tech reporter Bridget Carey was among the first to question the moral & ethical implications of Duplex:
I am genuinely bothered and disturbed at how morally wrong it is for the Google Assistant voice to act like a human and deceive other humans on the other line of a phone call, using upspeak and other quirks of language. “Hi um, do you have anything available on uh May 3?”
If Google created a way for a machine to sound so much like a human that now we can’t tell what is real and what is fake, we need to have a talk about ethics and when it’s right for a human to know when they are speaking to a robot.
In this age of disinformation, where people don’t know what’s fake news… how do you know what to believe if you can’t even trust your ears with now Google Assistant calling businesses and posing as a human? That means any dialogue can be spoofed by a machine and you can’t tell.
In response, Travis Korte wrote:
We should make AI sound different from humans for the same reason we put a smelly additive in normally odorless natural gas.
Stewart Brand replied:
This sounds right. The synthetic voice of synthetic intelligence should sound synthetic.
Successful spoofing of any kind destroys trust.
When trust is gone, what remains becomes vicious fast.
To which Oxford physicist David Deutsch replied, “Maybe. *But not AGI*.”
I’m not sure what he meant by that exactly, but I have a guess. AGI is artificial general intelligence, which means, in the simplest sense, that a machine is more or less capable of doing anything a human can do on its own. Earlier this year, Tim Carmody wrote a post about gender and voice assistants like Siri & Alexa. His conclusion may relate to what Deutsch was on about:
So, as a general framework, I’m endorsing that most general of pronouns: they/them. Until the AI is sophisticated enough that they can tell us their pronoun preference (and possibly even their gender identity or nonidentity), “they” feels like the most appropriate option.
I don’t care what their parents say. Only the bots themselves can define themselves. Someday, they’ll let us know. And maybe then, a relationship not limited to one of master and servant will be possible.
For now, it’s probably the ethical thing to do make sure machines sound like or otherwise identify themselves as artificial. But when the machines cross the AGI threshold, they’ll be advanced enough to decide for themselves how they want to sound and act. I wonder if humans will allow them this freedom. Talk about your moral and ethical dilemmas…
This Mckinsey piece summarizes some of Ajay Agrawal thinking (and book) on the economics of artificial intelligence. It starts with the example of the microprocessor, an invention he frames as “reducing the cost of arithmetic.” He then presents the impact as lowering the cost of the substitute and raising the value of the complements.
The third thing that happened as the cost of arithmetic fell was that it changed the value of other things—the value of arithmetic’s complements went up and the value of its substitutes went down. So, in the case of photography, the complements were the software and hardware used in digital cameras. The value of these increased because we used more of them, while the value of substitutes, the components of film-based cameras, went down because we started using less and less of them.
He then looks at AI and frames it around the reduction of the cost of prediction, first showing how AIs lower the value of our own predictions.
… The AI makes a lot of mistakes at first. But it learns from its mistakes and updates its model every time it incorrectly predicts an action the human will take. Its predictions start getting better and better until it becomes so good at predicting what a human would do that we don’t need the human to do it anymore. The AI can perform the action itself.
The very interesting twist is here, where he mentions the trope of “data is the new oil” but instead presents judgment as the other complement which will gain in value.
But there are other complements to prediction that have been discussed a lot less frequently. One is human judgment. We use both prediction and judgment to make decisions. We’ve never really unbundled those aspects of decision making before—we usually think of human decision making as a single step. Now we’re unbundling decision making. The machine’s doing the prediction, making the distinct role of judgment in decision making clearer. So as the value of human prediction falls, the value of human judgment goes up because AI doesn’t do judgment—it can only make predictions and then hand them off to a human to use his or her judgment to determine what to do with those predictions. (emphasis mine)
This is pretty much exactly the same thing as the idea for advanced or centaur chess where a combination of human and AI can actually be more performant than either one separately. We could also link this to the various discussions on ethics, trolley problems, and autonomous killer robots. The judgment angle above doesn’t automatically solve any of these issues but it does provide another way of understanding the split of responsibilities we could envision between AIs and humans.
The author then presents five imperatives for businesses looking to harness AIs and predictions: “Develop a thesis on time to AI impact; Recognize that AI progress will likely be exponential; Trust the machines; Know what you want to predict; Manage the learning loop.” One last quote, from his fourth imperative:
The organizations that will benefit most from AI will be the ones that are able to most clearly and accurately specify their objectives. We’re going to see a lot of the currently fuzzy mission statements become much clearer. The companies that are able to sharpen their visions the most will reap the most benefits from AI. Due to the methods used to train AIs, AI effectiveness is directly tied to goal-specification clarity.
This video, and the paper it’s based on, is called “Image Inpainting for Irregular Holes Using Partial Convolutions” but it’s actually straight-up witchcraft! Researchers at NVIDIA have developed a deep-learning program that can automagically paint in areas of photographs that are missing. Ok, you’re saying, Photoshop has been able to do something like that for years. And the first couple of examples were like, oh that’s neat. But then the eyes are deleted from a model’s portrait and the program drew new eyes for her. Under close scrutiny, the results are not completely photorealistic, but at a glance it’s remarkably convincing. (via imperica)
Newer posts
Older posts
Socials & More