8 Artificial Intelligence Image and Video Generators Built Using Stable Diffusion to Try Out
Contents
With the rise of artificial intelligence (AI), there has been a lot of discussion about what this could mean for the future. It has been playing an important role in image processing for a while but it's gaining a lot of attention recently following the release of Stable Diffusion, a text-to-image AI model by Stability AI.
The state-of-the-art AI model is capable of generating realistic photos and arts from any text input and we are seeing more projects built on top of it emerging to leverage the power of AI.
In this article, we have listed 8 interesting projects that are built using Stable Diffusion and they could be an inspiration for your next project.
1. DreamBooth
source: DreamBooth
Google’s DreamBooth is a fine-tuned text-to-image diffusion model that generates images based on a subject. Using typically 3-5 images that contain a subject and its class name (eg. dog, man, woman, etc.) as the input of a pre-trained model, a unique personalized text-to-image model will be trained.
The model will encode a unique identifier that refers to the subject. As a result, you can enter text prompts containing the subject's class name with other attributes to generate a variety of images of the subject in diverse scenes, poses, views, art styles, and lighting conditions that do not appear in the reference images.
By feeding the model with images of different subjects, you will get models that are personalized to each subject. Here are some of them that are generated from screenshots of famous movies, which you can use to generate images in the movies' styles:
Classic Disney Animation
source: Nitrosocke
Cyberpunk
source: DGSpitzer
Other than these trained models, there are also other projects that are built on top of DreamBooth:
And here are some images that are generated by Avatar AI and ProfilePicture.AI:
source: Jon Yongfook
2. Imagic
source: Imagic
Google’s Imagic is a text-based real image editing model that can edit images using text. It can perform highly complex image editing operations like changing posture and editing multiple objects just by using text prompts.
As showcased on the paper published, the results generated are mind-blowing. To achieve the same result by editing the photo manually using a photo editing software would require some skills and yet it can be done so easily by Imagic.
Not only that, the style of the image can also be changed:
source: Imagic
If you're interested to try it out, here’s a notebook that implements the Imagic model.
3. Stock AI
Different from usual stock photography platforms, Stock AI is 100% AI-powered. It has thousands of pre-generated stock photographs which you can search from its platform. As the images are generated by AI, they can be created based on your text input even if they don't exist on the platform.
One of the problems with normal stock photograph platforms is that the images are not exclusive. Using AI-generated stock photos, you can have original and unique high-quality images that cannot be found anywhere else, and even own the copyrights.
Results from searching “portrait”:
4. Lexica
Lexica does not only generate images, it's also a search engine for Stable Diffusion images and prompts, which makes it a great source of inspiration for AI image generation ideas. Every image on the platform has a prompt containing different keywords that can be copied and remixed, making constructing a prompt that generates similar images much easier.
Besides that, you can also upload an image to reverse search similar AI-generated images. On top of using the website to perform normal and reverse search, APIs are also available for free. Every API request will return an array of 50 images.
Search result using an uploaded image:
5. Stable Diffusion Infinity
source: Stable Diffusion Infinity
Powered by Stable Diffusion, the project enables users to outpaint and extend an image using AI-generated images. By entering text prompts which describe the context of the desired image for extension, the AI-generated images can blend really well with the original image.
The result is as shown in the image below:
source: Stable Diffusion Infinity
It is an open source project and its web app made using PyScript and Gradio is available for testing on Hugging Face.
6. Alpaca (Beta)
Alpaca is a Photoshop plugin that integrates Stable Diffusion into Photoshop to generate synthetic images. From the demonstration video posted by the creator, the plugin is capable of generating images in a very distinguishable style using text prompts with other inputs like a modifier, guidance scale, and steps without leaving Photoshop.
Its inpainting and outpainting features make replacing an object in an image and extending the image in Photoshop really easy. In the example shown below, the old man and the painting on the left bottom corner are replaced with an old lady and a new painting generated by the model using inpainting, while the right side of the image is extended using outpainting.
Before
source: Alpaca
After
source: Alpaca
7. Seamless Textures by Travis Hoppe
This project uses Stable Diffusion to generate seamless textures—images that have no visible boundaries or edges. These images are often used by 3D artists, CAD users, and designers to create ultra-realistic renders of products and 3D models. Textures that are often used include wood, grass, marble, and stone.
By slicing and masking the image, an infinitely tiled image can be created, meaning that the size of the image can be set infinitely. As these images have no visible boundaries, they are also often used as background images.
The method works for both real images and abstract images. The image below shows the result generated from an abstract image:
source: Travis Hoppe
8. Stable Diffusion Videos by Nate Raw
This project allows you to create AI music videos by interpolating the latent space of the text-to-image Stable Diffusion model. A video can be generated by morphing images created from the same prompt or multiple prompts provided that they use the same settings (guidance scale, scheduler, height, width).
After the video is created, you can load in an audio clip so that the video will move to the beat of the audio.
The creator of the project has written a comprehensive tutorial on how to create a video using the model and explains how it works very clearly. If you want to find out the hows and whys, you should definitely check it out.
The Future of AI-Generated Content
With the rapid growth of AI, it is no surprise that it is being used to produce content in a variety of ways. As the technology continues to evolve, we’ll probably see more and more projects built on top of Stable Diffusion and other AI models to use AI-generated content to its fullest potential.
