Abstract

This disclosure relates to methods, non-transitory computer readable media, and systems that can render a virtual object in a digital image by using a source-specific-lighting-estimation-neural network to generate three-dimensional ("3D") lighting parameters specific to a light source illuminating the digital image. To generate such source-specific-lighting parameters, for instance, the disclosed systems utilize a compact source-specific-lighting-estimation-neural network comprising both common network layers and network layers specific to different lighting parameters. In some embodiments, the disclosed systems further train such a source-specific-lighting-estimation-neural network to accurately estimate spatially varying lighting in a digital image based on comparisons of predicted environment maps from a differentiable-projection layer with ground-truth-environment maps.

IPC Classes  ?

• G06T 15/50 - Lighting effects
• G06T 1/40 - Neural networks

Abstract

Method for determining a normal map on the basis of a plurality of 2D images by means of uncalibrated photometric stereo, comprising the steps of: - receiving a set of images with various uncalibrated lighting conditions; - extracting a field of intermediate normals and a field of intermediate lights that result from decomposing the matrix I containing the image data into singular values; - determining six minor coefficients of the inverse of the linear transformation to be applied to field of normals in order to make it integrable; - modifying the field of normals in order to make its induced surface integrable; - extracting Lambertian diffuse reflectance maxima (LDRM); - determining the generalized bas-relief ambiguity based on the LDRM using a statistical approach; - applying the GBR to the fields of intermediate normals and of intermediate lights; - normalizing the normals of the field of intermediate normals.

IPC Classes  ?

• G06T 7/586 - Depth or shape recovery from multiple images from multiple light sources, e.g. photometric stereo