How generative AI is changing art and design


Image: Google

You've heard a lot about AI in art and design in 2017, with Adobe trumpeting its Sensei machine-learning technology for everything from image recognition in Lightroom to tech previews of turning photos into 'hand-drawn' sketches and animated graph creation at this year's Adobe Max conference.

But - from looking at what researchers at companies like Google and universities around the world are working on - this is just the beginning. Here James Kobielus, SiliconAngle Wikibon's lead analyst for AI, data science, and application development, takes a look at the latest imaging research that could affect how - and what - you create in the future.

A lot of the examples feature imagery that aren't what you'd call artistic - unsurprising as they're largely created by people from a science rather than arts background. They also often have a fractal distortion reminiscent of an acid trip (but that's just inherent in AI, it turns out, rather than the researchers behind them necessarily being Timothy Leary-types). And their research is often presented in the ultra-detailed format of scientific papers - heavy on words and, strangely to us, light on visual examples. But persevere - perhaps as reading over the Christmas break - and you'll find both insight and inspiration from the next generation of technology of art and design.

Generative AI: The new power tool for creative pros

Artificial intelligence can generate fresh patterns in data with astonishing speed, efficiency, and verisimilitude. Over the past few years, it has become commonplace for AI to algorithmically generate any object that can be rendered digitally.

Increasingly, this application of AI is being referred to as 'generative'. Generative AI will drive the next generation of apps for autoprogramming, content development, visual arts, and other creative, design, and engineering activities.

By 2019, most leading AI providers will offer tools and libraries for building AI-powered natural-language generation, image manipulation, and other generative use cases. Already, generative AI has proven itself - in both research and in commercial applications—in the following areas:

Generative graphics: AI can abstract visual patterns from artwork and then apply those patterns in the fanciful re-rendering of photographic images with the hallmark features of that artwork. These algorithms can also transform any rough doodle into an impressive drawing that seems to have been created by expert human artists depicting real-world models. They can take hand-drawn sketches of human faces and algorithmically transform them into photorealistic images (below).

They can instruct a computer to render any image so it appears as if it were composed by a specific human artist in a specific style. And they can organically conjure from any image any patterns, figures, and other details that were not present in the source.

Generative photos: AI can autocorrect photos by generating and superimposing onto the original any visual elements that were missing, obscure, or misleading. It can also transform any low-resolution original image into a natural-looking high-resolution version. It can generate natural-looking but synthetic human faces by blending existing portraits or abstracting features from any specific portrait. 

And it can generate photo-realistic images from semantic label maps.

Generative audio: AI can render any computer-generated voice into one that truly sounds like it was naturally produced in a human vocal tract. It can translate text to speech with astonishing naturalness. It can also compose music that feels like it expresses some authentic feeling deep in the soul of an actual human musician.

Generative video: AI can tweak frames in a motion picture to conform with whatever style, lighting, or other effect the director wants to achieve.

Generative text: AI can generate natural language content rapidly in large varieties at scale at top speed. It can generate captions, annotations, and other narratives from images and other content. And it can blend existing fonts into new font designs.

Generative code: AI can generate builds of program code that address an application domain of interest.

Generative materials: AI can even render prosthetic limbsorganic molecules, and other items from scratch when actuated through 3D printingCRISPR, and other technologies.

In 2018 and beyond, more solutions will come to market - in all verticals - that use leading-edge AI approaches known as generative adversarial networks (GANs) to algorithmically create digital and analogue objects of all sorts with astonishing accuracy.

Before 2019, more providers will roll out GAN-driven tools and workbenches for software programming, computer-aided design, web content development, music composition, image manipulation, video production, and other creative disciplines. And generative photo apps are likely to come to every smart camera application on mobile devices and other mass-market IoT endpoints.

GANs were a massive global research focus in the AI community in 2017. The pace of advances in GAN technology is likely to accelerate in the coming year. And generative design techniques are likely to come into the core curricula of data science, creative, and engineering professions globally.