8 Artificial Intelligence Image and Video Generators Built Using Stable Diffusion to Try Out

AI-generated content has been the talk of the town. With the release of Stable Diffusion, we are seeing more projects on AI-generated image and here are eight of them that you can't miss out.
by Josephine Loo · November 2022

Contents

    With the rise of artificial intelligence (AI), there has been a lot of discussion about what this could mean for the future. It has been playing an important role in image processing for a while but it's gaining a lot of attention recently following the release of Stable Diffusion, a text-to-image AI model by Stability AI.

    The state-of-the-art AI model is capable of generating realistic photos and arts from any text input and we are seeing more projects built on top of it emerging to leverage the power of AI.

    In this article, we have listed 8 interesting projects that are built using Stable Diffusion and they could be an inspiration for your next project.

    1. DreamBooth

    an example of how DreamBooth works source: DreamBooth

    Google’s DreamBooth is a fine-tuned text-to-image diffusion model that generates images based on a subject. Using typically 3-5 images that contain a subject and its class name (eg. dog, man, woman, etc.) as the input of a pre-trained model, a unique personalized text-to-image model will be trained.

    The model will encode a unique identifier that refers to the subject. As a result, you can enter text prompts containing the subject's class name with other attributes to generate a variety of images of the subject in diverse scenes, poses, views, art styles, and lighting conditions that do not appear in the reference images.

    By feeding the model with images of different subjects, you will get models that are personalized to each subject. Here are some of them that are generated from screenshots of famous movies, which you can use to generate images in the movies' styles:

    Classic Disney Animation

    images generated from DreamBooth in Disney style source: Nitrosocke

    Cyberpunk

    images generated from Cyberpunk in Disney style source: DGSpitzer

    Other than these trained models, there are also other projects that are built on top of DreamBooth:

    And here are some images that are generated by Avatar AI and ProfilePicture.AI:

    AI-generated images of Jon Yongfook source: Jon Yongfook

    2. Imagic

    an example of how Imagic works source: Imagic

    Google’s Imagic is a text-based real image editing model that can edit images using text. It can perform highly complex image editing operations like changing posture and editing multiple objects just by using text prompts.

    As showcased on the paper published, the results generated are mind-blowing. To achieve the same result by editing the photo manually using a photo editing software would require some skills and yet it can be done so easily by Imagic.

    Not only that, the style of the image can also be changed:

    an example of DreamBooth source: Imagic

    If you're interested to try it out, here’s a notebook that implements the Imagic model.

    3. Stock AI

    Stock AI

    Different from usual stock photography platforms, Stock AI is 100% AI-powered. It has thousands of pre-generated stock photographs which you can search from its platform. As the images are generated by AI, they can be created based on your text input even if they don't exist on the platform.

    One of the problems with normal stock photograph platforms is that the images are not exclusive. Using AI-generated stock photos, you can have original and unique high-quality images that cannot be found anywhere else, and even own the copyrights.

    Results from searching “portrait”:

    Results from searching “portrait”

    4. Lexica

    Lexica

    Lexica does not only generate images, it's also a search engine for Stable Diffusion images and prompts, which makes it a great source of inspiration for AI image generation ideas. Every image on the platform has a prompt containing different keywords that can be copied and remixed, making constructing a prompt that generates similar images much easier.

    Besides that, you can also upload an image to reverse search similar AI-generated images. On top of using the website to perform normal and reverse search, APIs are also available for free. Every API request will return an array of 50 images.

    Search result using an uploaded image:

    Search result using an uploaded image on Lexica

    5. Stable Diffusion Infinity

    Stable Diffusion Infinity: before source: Stable Diffusion Infinity

    Powered by Stable Diffusion, the project enables users to outpaint and extend an image using AI-generated images. By entering text prompts which describe the context of the desired image for extension, the AI-generated images can blend really well with the original image.

    The result is as shown in the image below:

    Stable Diffusion Infinity: after source: Stable Diffusion Infinity

    It is an open source project and its web app made using PyScript and Gradio is available for testing on Hugging Face.

    6. Alpaca (Beta)

    Alpaca Stable Diffusion plugin for Photoshop

    Alpaca is a Photoshop plugin that integrates Stable Diffusion into Photoshop to generate synthetic images. From the demonstration video posted by the creator, the plugin is capable of generating images in a very distinguishable style using text prompts with other inputs like a modifier, guidance scale, and steps without leaving Photoshop.

    Its inpainting and outpainting features make replacing an object in an image and extending the image in Photoshop really easy. In the example shown below, the old man and the painting on the left bottom corner are replaced with an old lady and a new painting generated by the model using inpainting, while the right side of the image is extended using outpainting.

    Before

    Alpaca: before source: Alpaca

    After

    Alpaca: after source: Alpaca

    7. Seamless Textures by Travis Hoppe

    This project uses Stable Diffusion to generate seamless textures—images that have no visible boundaries or edges. These images are often used by 3D artists, CAD users, and designers to create ultra-realistic renders of products and 3D models. Textures that are often used include wood, grass, marble, and stone.

    By slicing and masking the image, an infinitely tiled image can be created, meaning that the size of the image can be set infinitely. As these images have no visible boundaries, they are also often used as background images.

    The method works for both real images and abstract images. The image below shows the result generated from an abstract image:

    an abstract seamless texture image source: Travis Hoppe

    8. Stable Diffusion Videos by Nate Raw

    This project allows you to create AI music videos by interpolating the latent space of the text-to-image Stable Diffusion model. A video can be generated by morphing images created from the same prompt or multiple prompts provided that they use the same settings (guidance scale, scheduler, height, width).

    After the video is created, you can load in an audio clip so that the video will move to the beat of the audio.

    The creator of the project has written a comprehensive tutorial on how to create a video using the model and explains how it works very clearly. If you want to find out the hows and whys, you should definitely check it out.

    The Future of AI-Generated Content

    With the rapid growth of AI, it is no surprise that it is being used to produce content in a variety of ways. As the technology continues to evolve, we’ll probably see more and more projects built on top of Stable Diffusion and other AI models to use AI-generated content to its fullest potential.

    About the authorJosephine Loo
    Josephine is an automation enthusiast. She loves automating stuff and helping people to increase productivity with automation.

    Top 5 Automated UI Testing Tools to Make Testing Faster in 2022

    Speed is key in 2022, test and ship fast. These five tools will get your tests running faster than ever. Find out more about how they can help to make sure your application functions as expected in this article.

    How to Build a Video Thumbnail Generator Tool Using React and Bannerbear

    Improve users' workflow by integrating a video thumbnail generator that adds text to an image uploaded by them using Bannerbear API.

    How to Convert Audio to Video for Youtube Upload Using FFmpeg

    Learn how to convert music, podcasts, and audiobooks to videos with a still background image to upload these types of audio content to Youtube.

    Follow the Journey

    Sign up for our once a fortnight newsletter update on new Bannerbear features and our business journey

    8 Artificial Intelligence Image and Video Generators Built Using Stable Diffusion to Try Out
    8 Artificial Intelligence Image and Video Generators Built Using Stable Diffusion to Try Out