Nvidia's researchers released a deep learning method that can edit images or reconstruct damaged images, implementing P-maps with one key, and “no trace of psâ€. By using a "partial convolution" layer, this method is superior to other methods.
In the field of computer vision research, NVIDIA is often eye-catching.
Such as "training GAN with Progressive Growing, generating ultra-realistic high-definition images," "pix pix2pixHD project with image processing and processing at 2048x1024 resolution with conditional GAN", or brainstorming, letting the rain fall on a sunny day, and turning a kitten into a lion The "Unsupervised Image-to-Image Translation Networks" is the night-to-day "unsupervised image translation network."
Recently, NVIDIA released an ICLR this year on arXiv, which is also very powerful. The paper is titled "Image Inpainting for Irregular Holes Using Partial Convolutions", which uses "Partial Convolutions" for image restoration.
In the video on the left side of the interface, simply use the tool to simply erase the unwanted content of the image, even if the shape is very irregular, NVIDIA's model can "restore" the image, fill the blank with a very realistic picture . Can be described as a key P map, and "no trace of ps."
The study came from the team of Guilin Liu et al. of Nvidia. They published a deep learning method that can edit images or reconstruct damaged images, even if the image has holes or missing pixels. This is the current state-of-the-art approach.
The method can also achieve image editing by removing some of the content in the image and filling in the blank resulting from the removal of the content.
This process is called "image inpainting" and can be used to remove unwanted content in image editing software while filling in the gaps with computer-generated realistic alternatives.
Figure: Covered image, and repair results using a partial convolution based network
"Our model can handle any shape, size, position, or any distance from the edge of the image well. Previous deep learning methods focused on rectangular areas near the center of the image, and often rely on costly post processing. "Nvidia's researchers wrote in their research report, "In addition, our model can deal with larger and larger blank areas."
In order to train the neural network, the research team first generated 55,116 random color bars, masks of any shape and size for training. They also generated 25,000 images for testing. In order to improve the accuracy of reconstructed images, the researchers further divided these training images into 6 categories based on the size of the input images.
Figure: Some masks for testing
Using the NVIDIA Tesla V100 GPU and the cuDNN-accelerated PyTorch deep learning framework, the team trained its neural network by applying the generated mask to the images of the ImageNet dataset Places2 and CelebA-HQ.
Figure: Comparison of test results on ImageNet
Figure: Comparison of test results on Place2 dataset
During the training phase, blank or missing portions are introduced into the complete training image of the above data set so that the network can learn to reconstruct missing pixels.
During the testing phase, another blank or missing portion not used during training is introduced into the test image in the data set to perform unbiased verification of the accuracy of the reconstructed image.
Figure: Comparison of results (Conv) based on a typical convolutional layer and PConv (partial convolution) layers
The researchers stated that the existing deep learning based image restoration method is not good enough because the output of missing pixels necessarily depends on the value of the input, and these inputs must be provided to the neural network to find the missing pixels. This leads to artifacts such as color differences or blurring in the image.
To solve this problem, the NVIDIA team developed a way to ensure that the output of missing pixels does not depend on the value of the input provided for these pixels. This method uses a "partial convolution" layer, which renormalizes each output based on its validity for the corresponding receptive field. This renormalization ensures that the output value is independent of the value of missing pixels in each accepted domain.
The model is built using the UNet architecture implemented by these partial convolutions. A set of loss functions is used to match the feature loss and style loss of the VGG model and the model is trained to produce a realistic output.
Table: Comparison of results from various methods
The research team stated that the model is superior to the previous method.
"As far as we know, we were the first to demonstrate the effects of deep learning image restoration models on irregular shaped holes," said NVIDIA researchers.
The researchers also mentioned in the paper that the same framework can also be used to handle image super resolution tasks.
Best Budget I7 Laptop is everyone eager to. Laptop Intel Core i7 is the No. 1 processor clients choose when they are searching for a performance Gaming Laptop,cause it`s of excellent powering, bigger Solid State Drive, usually 512GB based, full HD 16:9 aspect ratio slimmer display bezels, bigger battery-12000mAh-no need to worry lack of power, fingerprint reader, backlight keyboard, etc. What is intel Core I7 Laptop Price? Usually 300-400usd. You can get Intel i7 11th Generation Laptop, Intel i7 10th Generation Laptop, Intel I5 11th Generation Laptop, Laptop I3 11th Generation,etc. 15.6 or 14 inch all available for i7 processor.
Cooperating with us, you just need to share all your requirements details, like size, cpu, ram, rom, battery, fingerprint, backlight, enter button numbers, SSD+HDD or only SSD ok, if need oem service, if need logo position, special apps preinstall if need, etc. Can provide matched and value feedback in 1-2 working days.
The reason why clients choose us is that just half of DOA than others, longer warranty time, smaller MOQ, fast delivery, competitive cost, good after-sale service, etc.
Laptop Intel Core I7,Intel Core I7 Laptop Price,Best Budget I7 Laptop,Intel I7 11th Generation Laptop,Intel I7 10th Generation Laptop
Henan Shuyi Electronics Co., Ltd. , https://www.shuyielectronictech.com