The AI research team at Qualcomm has recently made a significant breakthrough by deploying Stable Diffusion, a popular AI-based text-to-image generative model, on a smartphone using the company's AI Stack.
Stable Diffusion, which consists of more than one billion parameters, was previously restricted to cloud servers due to its size. However, by utilizing the Qualcomm AI Stack, researchers at Qualcomm AI Research were able to carry out full-stack AI optimizations to allow Stable Diffusion to be run on an Android smartphone.
According to the company, this is the first time Stable Diffusion has been deployed using a mobile device.
How Qualcomm Made This Possible
The company asserts that the Qualcomm AI Stack enables OEMs and developers to create, optimize, and deploy AI applications on Qualcomm's products while maximizing the performance of the Qualcomm AI Engine.
But how did they do it? Qualcomm AI Research started with the FP32 version 1-5 open-source model from Hugging Face, a data science platform. Researchers then made quantization, compilation, and hardware acceleration optimizations to make the Stable Diffusion model run on the latest Snapdragon 8 Gen 2 chipset operated in a mobile phone.
Researchers at Qualcomm AI used the AI Model Efficiency Toolkit's post-training quantization to shrink the model. Quantization improves performance and saves energy by enabling the model to run well on AI-specific hardware and use less memory bandwidth.
Read Also: Qualcomm Says US Ban Won't Affect its Supplying Chips to Huawei: Snapdragon Unaffected
The Qualcomm AI Engine direct framework then arranges the operations correctly. This helps improve performance and reduce memory leaks. These improvements arose because AI optimization researchers and compiler engineering teams worked together to improve how memory is managed in AI inference.
AI Art Generator Now Available on Android
As a result of this full-stack optimization, Stable Diffusion can now generate a 512x512 pixel image from 20 inference steps in less than 15 seconds on a smartphone, with completely unrestricted text input from the user. Check out this video from Qualcomm:
Qualcomm says this is the fastest inference time ever recorded on a smartphone and is comparable to cloud latency. Although Stable Diffusion is a large model, it encodes substantial knowledge about speech and visuals, allowing it to generate virtually any conceivable image.
As per ZDNet, Qualcomm will demonstrate Stable Diffusion running locally on a premium smartphone powered by the company's latest Snapdragon 8 Gen 2 chipset for the first time at Mobile World Congress (MWC).
Qualcomm claimed a speed record for its Stable Diffusion software, which it says has never run locally on Android before. However, The Verge reports that developer Ivon Huang has demonstrated that the technology can be run on a Sony Xperia 5 II using a Qualcomm Snapdragon 865 and 8GB of RAM. Huang noted that generating a 512 x 512 image took an hour.
Apple has also released the optimizations needed to run Stable Diffusion on its Core ML framework in about a minute. However, Qualcomm's software is still faster, taking just a minute to generate a 512 x 512 image on an iPhone 13.
Stay posted here at Tech Times.
Related Article: Why Stable Diffusion AI Lawsuit is More Than Copyright Infringement; Here's What Lawyer Reveals
