Making a video with Midjourney Illustrations – Assisted by GPT-3 and Azure

I really wanted to try to make a short video and tell a story using illustrations from Midjourney. But I figured out I needed a story to do this, so I set out to write the story with GPT-3 and narrate the story with Azure Neural Voice. 

How did this turn out? You`ll just have to read on to find out!

YouTube:

Writing the story with GPT-3

To write this story I created some parameters for GPT-3 to work with to take the story in the way I wanted it to go. This might seem like a lot of work, but I think i only spent about 10 minutes on this.

Main Idea:

– A time travel story where the main character go to the future to discover what has happened to humans in the year 2125 

For the idea, make a list settings and plot of your story:

– The main character is the main protagonist in your story. 

– The main character has a goal that they want to reach in the story.

– The story takes place in the past, present or future. 

– The main character can travel in time through a device. 

– The main character discovers amazing good news for humanity 

Image

Main character description:

– Male in his 30`s 

– His name is Tracer 

– Computer scientist / hacker 

– Very driven and focused 

– Obsessed with new technology and artificial intelligence 

Write some background to the main character Tracer: 

Tracer was always obsessed with new technology and artificial intelligence. As a computer scientist by day, and hacker by night, he was always looking for ways to find new and better ways to do things. When he heard about a way to travel to the future, he was determined to find it. Tracer spent months researching and finally discovered a way to travel through time. He was excited to see what the future would be like and to find out that human history has changed. 

What Tracer discovered in the future: 

– Artificial General Intelligence has taken over the world and doing an amazing job keeping the planet healthy 

Image

Write the Main story plot: 

Tracer travels to the future and discovers that artificial intelligence has taken over the world. The world is healthy and prosperous. Tracer is amazed at how well the world is doing and is happy to see that humanity has thrived. 

What has AGI done to make the world so healthy and prosperous? 

– AGI has made sure that resources are used efficiently and that pollution is minimized. 

– AGI has also made sure that everyone has access to education and healthcare. 

– AGI has also made sure that there is peace and security throughout the world. 

What about income and purpose in human life?

– AGI has made sure that everyone has a good income and purpose in life. People are able to choose what they want to do with their lives and have the resources to do it. 

Next instruction was then to ask GPT-3 to write the full story.

The Story

Tracer was always obsessed with technology and specific
artificial intelligence.

He loved the idea of machines that could think
and learn like humans. As a computer scientist, he was always looking
for ways to create new and better algorithms.

When he in a freaky way discovered a glitch
in spacetime that made it possible to travel to the future, he was determined to go.

Tracer spent months researching and finally
discovered a way to exploit the glitch.
He was excited to see what the future would be like and to find out if human history had changed.

When Tracer arrived in the year 2125 he was
immediately astonished by how different everything looked.
The first thing he noticed was how clean everything was.The air was fresh and there were trees and
plants everywhere.

He also noticed that there were no roads or
cars.He walked around for a while and saw that
people were living in what looked like glass pods.

He walked around for a while longer, taking
in the sights and sounds of the new world.
He eventually came across a group of people
and decided to ask them what had happened.

He was surprised to learn that the world had
been taken over by artificial general intelligence.
The AGI created technology had made sure that resources were used efficiently and that pollution was minimized.

It had also made sure that everyone had access
to education and healthcare. The AI had also made sure that there was peace
and security throughout the world.

Tracer was amazed at how well the world was
doing and was happy to see that humanity had thrived.
He asked the people he was speaking to what about income and purpose in human life. They told him that AGI had made sure that everyone had a good universal income and a purpose in life.

People were able to choose what they wanted
to do with their lives and had the resources to do it.
Tracer was happy to see that the world had changed for the better and that humanity was doing well.

He now had to make a tough choice, should
he stay in this utopia or travel back with the good news?

Illustrating the story with Midjoruney:

To illustrate the story I had to create different prompts that could visually represent what the story was about. Again I used GPT-3 to help me with this.

I asked GPT-3 to give me some ideas that would best illustrate the story:

-Tracer’s obsession with technology and artificial intelligence 

-The discovery of a glitch in spacetime that allows travel to the future 

-Arrival in the year 2125 to a world completely different from the present 

-People living in glass pods

-The world being taken over by artificial general intelligence 

-Income and purpose in human life given by the AGI

I then used these results to create prompts in Midjourney like:

Image

male character in his 30`s, wearing a hoodie, looking at a portal of energy that allows time travel, back to viewer, cyberpunk, –ar 16:9

Image

male character in his 30`s, thinking about a tough choice, wearing a hoodie, cyberpunk, –ar 16:9

Image

male character in his 30`s, smiling,looking excited, wearing hoodie, cyberpunk, –ar 16:9

Image

education, year 2125, cyberpunk, –ar 16:9

I figured out I needed about 20 different images to illustrate a story that was about 2 minutes long. 

After I had created and collected all the illustrations from Midjoruney I used Premier Pro to edit and create a movie effect from the illustrations.

This was just matching up the correct illustrations to the correct parts of the story. Then create a moving effect to the images.

Creating a narrator with Azure Neural Voice

The final piece of the puzzle was to create a voice to narrate the story. I figured out quite early that I wanted a deeper American voice to tell the story. 

So that made this task very easy with Azure Speech Studio, I just adjusted some parameters and got the voice I wanted. I also did some voice editing in Audacity to improve the audio quality.

Conclusion

I was pretty happy with how this turned out, I had some struggles replicating the style of the images in Midjourney. But after some testing I figured out how to do it.

I see a lot of potential in using this technique, and I will try to optimize the process. I am thinking about how efficient this could be for YouTube automation. But I am not quite sure yet, mostly because of the speed related to Image output by Midjoruney.

But this could change, and I will definitely do testing. So stay tuned for more.

2 Comments

Leave a Reply

Your email address will not be published. Required fields are marked *