AI (Artificial Intelligence) based upscalers in your web browser - 09/06/2022 - Written by Did

Tags
Image
Cat in the shades of Delphi, Greece

What is an upscaler ?

A good upscaling algorithm is processing a digital image for raising its resolution in such a way that quality is improved, to enable printing or displaying them on bigger paper and screens. Here is a summary of some projects you can use straight away or in your website.

In browser upscaling

Most up-scalers made for web embedding you can try directly in your browser from the demo page of their open source projects.

I call this practical, you can use them with your actual image, but try to keep the initial resolution really low or they may crash.

Yayoi-JS

While quite dated and not maintained (2018), this project works well with the Tensorflow.js library, it will double the size of the image you upload. Thanksfully it is client side which means it never leaves your computer :)

The project owner has made another attempt at upscaling with and algorithm specifically tailored for "2D" anime images (https://github.com/nagadomi/waifu2x).

Demo: https://panepo.github.io/Yayoi-js

Source code: https://github.com/Panepo/Yayoi-js

Image upscaling with Tensorflow.js

Use the slider to compare both side
photo after edit
photo before edit

UpscalerJS

This an actively maintained tensorflowjs based project which use a esrgan model, it is also very well documented on the author's blog.

Results are pretty good although the processing is quite slow as it requires much processing power.

Source: https://github.com/thekevinscott/UpscalerJS

Demo: https://www.upscaler.ai/

Keras-JS

Keras-JS works client side as well, it's a bit more dated than Yayoi (2017 vs 2018) and also not maintained, but it has the ability to use your GPU power (Nvidia RTX series are especially good at this), and the interface is more polish with multiple algorithm choices you can try.

Plus Keras-JS not only serve upscaling purpose but also a full suite of AI tools for image recognition (classification / categorizing objects).

Ironically, Keras-JS project moved to tensorflow.js but in 2022, there is still no official upscaling demo available, while Yayoi-JS actually made use of the newer library, I prefer the older, obsolete, Keras-JS one. The big downside is that with this demo, you can only use an image URL publicly available on the web, or you can convert your local image to a base64 encoded one with tools such as https://www.base64-image.de (then your url string must start with data:image/jpeg;base64,).

Demo: https://transcranial.github.io/keras-js/#/image-super-resolution

Source code: https://github.com/transcranial/keras-js

super-resolution-js

This one is the most recent project (april 2022), but it looks sketchy and is slow. The project relies an external library maintained by Microsoft, called ONNX. It is also totally unusable with my Firefox 103.0.1 ...

Demo: https://josephrocca.github.io/super-resolution-js/

Source code: https://github.com/josephrocca/super-resolution-js

Other projects you can try

More web based projects are available, the above are actual library which could theoretically be used in a website project, they consume your local computer resources (in another word the processing is done on "client side").

Below I will share with you some more projects you can use as upscalers, at no cost in your web browser, although they are using a remote backend for computing, and these dedicated resources might not stay available for free in the future:

Arc tencent

https://arc.tencent.com/en/ai-demos/humansegmentation

This awesome project does face restoration, people extraction for layering with alpha channel, and upscaling.

RealESRGAN

https://replicate.com/xinntao/realesrgan

Another esrgan implementation