DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Amendment
Applicant submitted amendments on 9/2/2026. The Examiner acknowledges the amendment and has reviewed the claims accordingly.
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The IDS(s) dated 04/09/2026, 02/06/2025, and 08/08/2024 that have been previously considered remain placed in the application file.
Overview
Claims 1-4 and 6-18 are pending in this application and have been considered below.
Claim 5 is canceled by the applicant.
Claims 1-4 and 6-18 are rejected.
Applicant Arguments
In regards to Argument 1, Applicant states that Fang’s appearance consistency heatmap measures local texture and appearance similarity rather than providing an ROI mask obtained from segmentation masks of categories depicting contextual information (See Remarks, page 11).
In regards to Argument 2, Applicant states that the cited references do not teach the amended limitation of obtaining the segmentation masks using a multi-class segmentation network configured to extract binary masks of the ROIs from the artifact free image (See Remarks, page 11-12).
Examiner’s Response
In response to Argument 1, it has been considered but is moot in view of new ground(s) of rejection based on the amendments. A new reference, Pugh, has been introduced which in ¶81 discloses each pixel associated with an object class can be combined to form a single object mask. Pugh introduces a similar process as described in the amended claim where category based masks are obtained from an input image and semantic segmentation identifies valid placement locations (¶¶78, 80-81, 165). In combination with the new reference Zhang, that processing determines the receiving region for the transformed shadow mask used in Zhang’s image synthesis framework.
After reviewing the amendments, the Examiner interprets that Zhang in view of Pugh teaches on the amended claims that were presented. The details of the rejection are listed below.
In response to Argument 2, it has been considered but is moot in view of new ground(s) of rejection based on the amendments. A new reference, Pugh, has been introduced which in ¶145 discloses semantic segmentation information can be a bitmask. Pugh introduces a similar process as described in the amended claim where neural network segmentation supplies per pixel class assignments and class derived masks with binary mask representations (¶¶78, 80-81, 139, 145). In combination with the new reference Zhang, this segmentation and mask output processing is incorporated into Zhang’s image synthesis framework.
After reviewing the amendments, the Examiner interprets that Zhang in view of Pugh teaches on the amended claims that were presented. The details of the rejection are listed below.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-2, 8-13, and 16-17 are rejected under 35 U.S.C. 103 as obvious over Zhang et al (Zhang, X., Barron, J. T., Tsai, Y. T., Pandey, R., Zhang, X., Ng, R., & Jacobs, D. E. (2020). Portrait shadow manipulation. ACM Transactions on Graphics (TOG), 39(4), 78-1, hereafter referred to as Zhang) in view of Pugh et al (US 20210142497 A1, hereafter referred to as Pugh).
Claim 1
Regarding Claim 1, Zhang teaches “A method for controlling an electronic device, the method comprising:
obtaining one or more artifact free images (Zhang in §3.1 discloses selecting portraits manuals identified as containing no foreign shadows) and one or more artifacts represented by one or more artifact masks (Zhang in §3.1 discloses selecting silhouette masks representing the shape of synthetic shadows);
obtaining a region of interest (ROI) mask identifying one or more ROIs in the one or more artifact free images;
generating one or more transformed artifact masks by applying at least one localization parameter amongst a plurality of localization parameters to the one or more artifact masks (Zhang in §3.1 discloses randomly scaling and tiling silhouette shadow masks);
generating one or more localized artifact masks by placing the one or more transformed artifact masks on the ROI mask (Zhang in §3.1 discloses transformed silhouette shadow masks); and
generating one or more artifact images by combining the one or more artifact free images and the one or more localized artifact masks (Zhang in §3.1, Equation (2), Figure 2 discloses combining the original image and its color varied counterpart through a shadow mask),
wherein the obtaining the ROI mask comprises:
obtaining, from the one or more artifact free images, one or more segmentation masks of one or more categories depicting contextual information associated with the one or more of ROIs; and
obtaining the ROI mask based on the one or more segmentation masks of the one or more categories,
wherein the one or more segmentation masks are obtained using a multi-class segmentation network configured to extract binary masks of the one or more ROIs from the one or more artifact free images.”
Zhang does not explicitly teach all of “… obtaining a region of interest (ROI) mask identifying one or more ROIs in the one or more artifact free images; generating one or more transformed artifact masks by applying at least one localization parameter amongst a plurality of localization parameters to the one or more artifact masks; generating one or more localized artifact masks by placing the one or more transformed artifact masks on the ROI mask; and … wherein the obtaining the ROI mask comprises: obtaining, from the one or more artifact free images, one or more segmentation masks of one or more categories depicting contextual information associated with the one or more of ROIs; and obtaining the ROI mask based on the one or more segmentation masks of the one or more categories, wherein the one or more segmentation masks are obtained using a multi-class segmentation network configured to extract binary masks of the one or more ROIs from the one or more artifact free images.”
However, Pugh teaches “… obtaining a region of interest (ROI) mask identifying one or more ROIs in the one or more artifact free images (Pugh in ¶¶78, 80-81 discloses obtaining semantic class masks from the input image; ¶¶139, 145 discloses binary mask and bitmask implementations; ¶165 discloses using semantic segmentation to identify valid placement locations);
generating one or more transformed artifact masks by applying at least one localization parameter amongst a plurality of localization parameters to the one or more artifact masks (Pugh in ¶¶158-165 discloses movement and positioning of inserted content);
generating one or more localized artifact masks by placing the one or more transformed artifact masks on the ROI mask (Pugh in ¶¶145-148, 158-165 discloses mask controlled rendering and semantically valid placement); and …
wherein the obtaining the ROI mask comprises:
obtaining, from the one or more artifact free images, one or more segmentation masks of one or more categories depicting contextual information associated with the one or more of ROIs (Pugh in ¶¶78, 80-81 discloses obtaining per pixel semantic object classes from the input image and combining pixels associated with a class into an object mask. ¶¶23, 165 discloses the category of the selected region supplies contextual information used to identify an appropriate receiving region); and
obtaining the ROI mask based on the one or more segmentation masks of the one or more categories (Pugh in ¶81 discloses class derived masks; ¶¶145-148 discloses using semantic masks to control image rendering; ¶165 discloses identifying valid placement locations through semantic segmentation),
wherein the one or more segmentation masks are obtained using a multi-class segmentation network configured to extract binary masks of the one or more ROIs from the one or more artifact free images (Pugh in ¶78 discloses neural network and CNN semantic segmentation of the input image; ¶80 discloses scores for multiple classes and pixel class selection; ¶81 discloses forming masks from class pixels; ¶¶139, 145 discloses binary mask and bitmask representations).”
Pugh is analogous art because Pugh, like Zhang, is in the field of localized synthesis and modification of images and is reasonably pertinent to the problem of placing synthetic content in an appropriate receiving region.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the shadow mask localization processing of Zhang with the image derived semantic class masks and region valid placement processing of Pugh because Pugh teaches that semantic segmentation and mask controlled rendering provides identification of valid placement locations and control over the regions receiving rendered content (¶¶145-148, 165), and one of ordinary skill would have recognized that incorporating this feature into the method of Zhang would produce a transformed shadow mask localized according to a selected category region of the same input image, with a reasonable expectation of success because both methods operate on image aligned pixel masks and the selected placement constraint can be applied before Zhang’s disclosed mask based blending without changing the shadow formation equation.
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Claim 2
Regarding Claim 2, Zhang in view of Pugh teaches The method as claimed in claim 1, further comprising:
generating one or more varied intensity images by applying one or more illumination parameters to the one or more artifact free images (Zhang in §3.1 discloses applying a randomly selected color correction matrix to the lit image to generate the color varied shadowed image); and
wherein the generating the one or more artifact images comprises:
generating the one or more artifact images by combining the one or more artifact free images, the one or more localized artifact masks and the one or more varied intensity images (Zhang in §3.1, Equation (2) discloses blending the original image and the color varied image through the shadow mask).
Claim 8
Regarding Claim 8, Zhang in view of Pugh teaches The method as claimed in claim 1, wherein the plurality of localization parameters comprises at least one of a translation, a rotation, a shear, a flip, and a scaling associated with the one or more artifact masks (Zhang in §3.1 discloses randomly scaled silhouette masks).
Claim 9
Regarding Claim 9, Zhang in view of Pugh teaches The method as claimed in claim 1, wherein the one or more artifacts comprises a region of shadow casted by one or more of an object, a scene, and a living being (Zhang in §1 discloses foreign shadows cast by external occluder, including trees, hat brims, an adjacent person, the camera, etc.) and the one or more artifact masks represent a location of the one or more artifacts using a binary mask image (Zhang in §3.1 discloses M=1 for fully shadowed and M=0 for fully lit pixels and uses silhouette input masks).
Claim 10
Regarding Claim 10, Zhang in view of Pugh teaches The method as claimed in claim 1, wherein the one or more artifact free images capture an object (Zhang in §3.1 discloses an image dataset identified as not containing any foreign shadows includes accessories, such as hats and scarves), a scene (Zhang in §3.1 discloses images include an environment ), and a living being (Zhang in §3.1 discloses images include a subject).
Claim 11
Regarding Claim 11, Zhang in view of Pugh teaches The method as claimed in claim 1,
wherein the one or more artifacts represented by the one or more artifact masks (Zhang in §3.1 discloses obtaining selected silhouette masks from a dataset. Pugh in ¶¶32, 36, 139 discloses device local repositories and mask/image storage for later processing) and the one or more artifact free images are pre-stored in the electronic device (Zhang in §3.1 discloses obtaining selected foreign shadow free portraits from a dataset. Pugh in ¶¶36, 139 discloses storing the input image in the user device for processing).
Claim 12
Regarding Claim 12, Zhang teaches “An electronic device comprising:
a memory storing one or more instructions (Zhang in §3.1 discloses obtaining selected silhouette masks from a dataset.); and
a processor (Zhang in §3.1 discloses obtaining selected silhouette masks from a dataset.) configured to execute the one or more instructions to:
obtain, from the memory, one or more artifact free images (Zhang in §3.1 discloses selecting portraits manuals identified as containing no foreign shadows) and one or more artifacts represented by one or more artifact masks (Zhang in §3.1 discloses selecting silhouette masks representing the shape of synthetic shadows);
obtaining a region of interest (ROI) mask identifying one or more ROIs in the one or more artifact free images;
generate one or more transformed artifact masks by applying at least one localization parameter amongst a plurality of localization parameters to the one or more artifact masks (Zhang in §3.1 discloses randomly scaling and tiling silhouette shadow masks);
generate one or more localized artifact masks by placing the one or more transformed artifact masks on the ROI mask (Zhang in §3.1 discloses transformed silhouette shadow masks); and
generate one or more artifact images by combining the one or more artifact free images and the one or more localized artifact masks (Zhang in §3.1, Equation (2), Figure 2 discloses combining the original image and its color varied counterpart through a shadow mask),
wherein the obtaining the ROI mask comprises:
obtaining, from the one or more artifact free images, one or more segmentation masks of one or more categories depicting contextual information associated with the one or more of ROIs; and
obtaining the ROI mask based on the one or more segmentation masks of the one or more categories,
wherein the one or more segmentation masks are obtained using a multi-class segmentation network configured to extract binary masks of the one or more ROIs from the one or more artifact free images.”
Zhang does not explicitly teach all of “… obtaining a region of interest (ROI) mask identifying one or more ROIs in the one or more artifact free images; generate one or more transformed artifact masks by applying at least one localization parameter amongst a plurality of localization parameters to the one or more artifact masks; generate one or more localized artifact masks by placing the one or more transformed artifact masks on the ROI mask; and … wherein the obtaining the ROI mask comprises: obtaining, from the one or more artifact free images, one or more segmentation masks of one or more categories depicting contextual information associated with the one or more of ROIs; and obtaining the ROI mask based on the one or more segmentation masks of the one or more categories, wherein the one or more segmentation masks are obtained using a multi-class segmentation network configured to extract binary masks of the one or more ROIs from the one or more artifact free images.”
However, Pugh teaches “… obtaining a region of interest (ROI) mask identifying one or more ROIs in the one or more artifact free images (Pugh in ¶¶78, 80-81 discloses obtaining semantic class masks from the input image; ¶¶139, 145 discloses binary mask and bitmask implementations; ¶165 discloses using semantic segmentation to identify valid placement locations);
generate one or more transformed artifact masks by applying at least one localization parameter amongst a plurality of localization parameters to the one or more artifact masks (Pugh in ¶¶158-165 discloses movement and positioning of inserted content);
generate one or more localized artifact masks by placing the one or more transformed artifact masks on the ROI mask (Pugh in ¶¶145-148, 158-165 discloses mask controlled rendering and semantically valid placement); and …
wherein the obtaining the ROI mask comprises:
obtaining, from the one or more artifact free images, one or more segmentation masks of one or more categories depicting contextual information associated with the one or more of ROIs (Pugh in ¶¶78, 80-81 discloses obtaining per pixel semantic object classes from the input image and combining pixels associated with a class into an object mask. ¶¶23, 165 discloses the category of the selected region supplies contextual information used to identify an appropriate receiving region); and
obtaining the ROI mask based on the one or more segmentation masks of the one or more categories (Pugh in ¶81 discloses class derived masks; ¶¶145-148 discloses using semantic masks to control image rendering; ¶165 discloses identifying valid placement locations through semantic segmentation),
wherein the one or more segmentation masks are obtained using a multi-class segmentation network configured to extract binary masks of the one or more ROIs from the one or more artifact free images (Pugh in ¶78 discloses neural network and CNN semantic segmentation of the input image; ¶80 discloses scores for multiple classes and pixel class selection; ¶81 discloses forming masks from class pixels; ¶¶139, 145 discloses binary mask and bitmask representations).”
Pugh is analogous art because Pugh, like Zhang, is in the field of localized synthesis and modification of images and is reasonably pertinent to the problem of placing synthetic content in an appropriate receiving region.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the shadow mask localization processing of Zhang with the image derived semantic class masks and region valid placement processing of Pugh because Pugh teaches that semantic segmentation and mask controlled rendering provides identification of valid placement locations and control over the regions receiving rendered content (¶¶145-148, 165), and one of ordinary skill would have recognized that incorporating this feature into the method of Zhang would produce a transformed shadow mask localized according to a selected category region of the same input image, with a reasonable expectation of success because both methods operate on image aligned pixel masks and the selected placement constraint can be applied before Zhang’s disclosed mask based blending without changing the shadow formation equation.
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Claim 13
Regarding Claim 13, Zhang in view of Pugh teaches The method as claimed in claim 1, further comprising:
generate one or more varied intensity images by applying one or more illumination parameters associated to the one or more artifact free images (Zhang in §3.1 discloses applying a randomly selected color correction matrix to the lit image to generate the color varied shadowed image); and
generate the one or more artifact images by combining the one or more artifact free images, the one or more localized artifact masks and the one or more varied intensity images (Zhang in §3.1, Equation (2) discloses blending the original image and the color varied image through the shadow mask).
Claim 16
Regarding Claim 16, Zhang teaches “A non-transitory computer-readable storage medium storing a program that is executable by a processor to perform a method for controlling an electronic device, the method comprising:
obtaining an artifact free image (Zhang in §3.1 discloses selecting portraits manuals identified as containing no foreign shadows) and an artifact mask identifying an artifact (Zhang in §3.1 discloses selecting silhouette masks representing the shape of synthetic shadows);
obtaining a localized artifact mask by applying at least one localization parameter to the artifact mask (Zhang in §3.1 discloses random scaling/tiling of silhouette masks)
obtaining an artifact image by combining the artifact free image with the localized artifact mask (Zhang in §3.1, Figure 2, Equation (2) discloses blending the original image and its color transformed counterpart through a shadow mask) … .”
Zhang does not explicitly teach all of “… obtaining a localized artifact mask by applying at least one localization parameter to the artifact mask and adjusting a position of the artifact mask to overlap with a region of interest (ROI) included in the artifact free image, wherein the ROI is determined based on one or more segmentation masks of one or more categories depicting contextual information; … wherein the one or more segmentation masks are obtained using a multi-class segmentation network configured to extract binary masks of the one or more ROIs from one or more artifact free images.”
However, Pugh teaches “… obtaining a localized artifact mask by applying at least one localization parameter to the artifact mask (Pugh in ¶¶158-165 discloses movement/positioning operations) and adjusting a position of the artifact mask to overlap with a region of interest (ROI) included in the artifact free image (Pugh in ¶¶158-165 discloses position adjustment and semantically valid placement),
wherein the ROI is determined based on one or more segmentation masks of one or more categories depicting contextual information (Pugh in ¶¶78, 80-81, 145, 165 discloses image derived semantic class masks and category informed placement regions); …
wherein the one or more segmentation masks are obtained using a multi-class segmentation network configured to extract binary masks of the one or more ROIs from one or more artifact free images (Pugh in ¶78 discloses a neural network/CNN semantic segmentation option operating on the raw input; ¶80 discloses multiple class scores and pixel class selection; ¶81 discloses class pixels forming object masks; ¶¶139, 145 discloses binary mask/bitmasks).”
Pugh is analogous art because Pugh, like Zhang, is in the field of localized synthesis and modification of images and is reasonably pertinent to the problem of placing synthetic content in an appropriate receiving region.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the shadow mask localization processing of Zhang with the image derived semantic class masks and region valid placement processing of Pugh because Pugh teaches that semantic segmentation and mask controlled rendering provides identification of valid placement locations and control over the regions receiving rendered content (¶¶145-148, 165), and one of ordinary skill would have recognized that incorporating this feature into the method of Zhang would produce a transformed shadow mask localized according to a selected category region of the same input image, with a reasonable expectation of success because both methods operate on image aligned pixel masks and the selected placement constraint can be applied before Zhang’s disclosed mask based blending without changing the shadow formation equation.
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Claim 17
Regarding Claim 17, Zhang in view of Pugh teaches The non-transitory computer-readable storage medium as claimed in claim 16, wherein the at least one localization parameters comprises at least one of a translation, a rotation, a shear, a flip, and a scaling associated with the one or more artifact masks (Zhang in §3.1 discloses randomly scaled silhouette masks).
Claims 3, 6-7, 14, and 18 are rejected under 35 U.S.C. 103 as obvious over Zhang et al (Zhang, X., Barron, J. T., Tsai, Y. T., Pandey, R., Zhang, X., Ng, R., & Jacobs, D. E. (2020). Portrait shadow manipulation. ACM Transactions on Graphics (TOG), 39(4), 78-1, hereafter referred to as Zhang and Pugh et al (US 20210142497 A1, hereafter referred to as Pugh), further in view of Li et al (CN113469929A, hereafter referred to as Li).
Claim 3
Regarding Claim 3, Zhang in view of Pugh teaches “The method as claimed in claim 1, further comprising:
analyzing the one or more artifact images to detect a presence of the one or more artifacts represented by the one or more localized artifact masks in the one or more artifact images (Pugh in ¶¶198-199 discloses detecting and isolating cast shadows, including with trained networks);
determining that the one or more artifacts represented by the one or more localized artifact masks is present in the one or more artifact images (Pugh in ¶199 discloses identifying shadow regions); and
producing the one or more artifact images comprising the one or more artifacts represented by the one or more localized artifact masks as an output (Zhang in §3.1, Figure 2 discloses producing shadow containing synthetic training images).”
Zhang in view of Pugh does not explicitly teach all of “… analyzing the one or more artifact images to detect a presence of the one or more artifacts represented by the one or more localized artifact masks in the one or more artifact images; determining that the one or more artifacts represented by the one or more localized artifact masks is present in the one or more artifact images; and producing the one or more artifact images comprising the one or more artifacts represented by the one or more localized artifact masks as an output.”
However, Li teaches “… analyzing the one or more artifact images to detect a presence of the one or more artifacts represented by the one or more localized artifact masks in the one or more artifact images (Li in ¶¶87-90 discloses validating the synthesized image after composition. The proposed modification applies Pugh’s shadow detector within the output validation stage to analyze Zhang’s generated image for the shadow represented by the localized mask);
determining that the one or more artifacts represented by the one or more localized artifact masks is present in the one or more artifact images (Li in ¶¶87-90, discloses determining whether the synthesized result satisfies the output validation condition. In the proposed modification, detecting the generated shadow at the localized mask region supplies the presence determination); and
producing the one or more artifact images comprising the one or more artifacts represented by the one or more localized artifact masks as an output (Li in ¶87-90 discloses retaining a successfully synthesized composite. The proposed combination outputs the generated shadow image following the presence determination).”
Li is analogous art because Li, like Zhang in view of Pugh, is in the field of computational image synthesis and is reasonably pertinent to the problem of determining whether a synthesized image contains the intended inserted content.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify output processing of Zhang in view of Pugh with the synthesized result validation/verification and positive result retention of Li because Li teaches that examining the synthesized result provides a determination that the intended content has been sufficiently incorporated (¶¶87-90), and one of ordinary skill would have recognized that incorporating this feature into the method of Zhang in view of Pugh would identify and retain generated images containing the intended shadow at the localized mask region, with a reasonable expectation of success because the generator produces an image and expected mask region suitable for image based shadow detection without altering the synthesis mechanism.
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Claim 6
Regarding Claim 6, Zhang in view of Pugh teaches “The method as claimed in claim 1”.
Zhang in view of Pugh does not explicitly teach all of “… wherein the plurality of localization parameters is determined based on: computing a size and dimensions associated with the ROI mask; and adjusting a plurality of pre-defined localization parameters of the one or more artifact masks according to the size and the dimensions of the ROI mask and generating the plurality of localization parameters.”
However, Li teaches “wherein the plurality of localization parameters is determined based on:
computing a size and dimensions associated with the ROI mask (Li in ¶¶n0081-n0091 discloses dimensionally comparing/fitting an inserted clothing instance to the receiving arm region, defined as its minimum bounding rectangle); and
adjusting a plurality of pre-defined localization parameters of the one or more artifact masks according to the size and the dimensions of the ROI mask and generating the plurality of localization parameters (Li in ¶¶82-85 discloses predefined scale factors and destination/source dimensional ratio scaling; ¶61 discloses transforming masks together with content; ¶¶n0100-201 discloses fitting to a particular masked receiving region).”
Li is analogous art because Li, like Zhang in view of Pugh, is in the field of mask based synthetic image generation and is reasonably pertinent to the problem of adapting the size and placement of inserted image content to dimensions of a receiving region.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the localization parameter determination for transforming artifact masks of Zhang in view of Pugh with the receiving region dependent scaling and dimension based placement adjustment of Li because Li teaches that dimension dependent scaling and placement provides appropriately fitted image content and reduces gaps at insertion boundaries (¶n0110), and one of ordinary skill would have recognized that incorporating this feature into the method of Zhang in view of Pugh would determine the size and dimensions of the ROI mask and adjust the predefined scaling and placement parameters of the artifact masks according to those dimensions, with a reasonable expectation of success because Zhang already applies geometric transformations to artifact masks, Pugh supplies the image aligned receiving region masks, and Li teaches destination relative scaling, transformation, and dimension based placement adjustments.
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Claim 7
Regarding Claim 7, Zhang in view of Pugh teaches The method as claimed in claim 6, further comprising:
varying the plurality of pre-defined localization parameters of the one or more artifact masks within a range to generate at least one other set of plurality of localization parameters (Li in ¶¶n0070-n0071, n0079 discloses prescribed random factor/rotation ranges, joint mask transformations and repeated synthesis);
applying the at least one other set of the plurality of localization parameters on the one or more artifact masks to generate at least one other transformed artifact mask (Li in ¶n0071 discloses applying the same scale/rotation to the mask and content; ¶n0130 discloses repeating synthesis. Zhang in §3.1 discloses varied silhouette masks); and
generating at least one other localized artifact mask by placing the at least one other transformed artifact mask on the ROI mask for further generating at least one other artifact image (Pugh in ¶165 discloses semantic region placement. Li in ¶¶n0071, n0130 discloses supplying repeated transformed mask placement/composition).
Claim 14
Regarding Claim 14, Zhang in view of Pugh teaches “The electronic device as claimed in claim 12, wherein the processor is further configured to:
analyze the one or more artifact images to detect a presence of the one or more artifacts represented by the one or more localized artifact masks in the one or more artifact images (Pugh in ¶¶198-199 discloses detecting and isolating cast shadows, including with trained networks);
determine that the one or more artifacts represented by the one or more localized artifact masks is present in the one or more artifact images (Pugh in ¶199 discloses identifying shadow regions); and
produce the one or more artifact images comprising the one or more artifacts represented by the one or more localized artifact masks as an output (Zhang in §3.1, Figure 2 discloses producing shadow containing synthetic training images).”
Zhang in view of Pugh does not explicitly teach all of “… analyze the one or more artifact images to detect a presence of the one or more artifacts represented by the one or more localized artifact masks in the one or more artifact images; determine that the one or more artifacts represented by the one or more localized artifact masks is present in the one or more artifact images; and produce the one or more artifact images comprising the one or more artifacts represented by the one or more localized artifact masks as an output.”
However, Li teaches “… analyze the one or more artifact images to detect a presence of the one or more artifacts represented by the one or more localized artifact masks in the one or more artifact images (Li in ¶¶87-90 discloses validating the synthesized image after composition. The proposed modification applies Pugh’s shadow detector within the output validation stage to analyze Zhang’s generated image for the shadow represented by the localized mask);
determine that the one or more artifacts represented by the one or more localized artifact masks is present in the one or more artifact images (Li in ¶¶87-90, discloses determining whether the synthesized result satisfies the output validation condition. In the proposed modification, detecting the generated shadow at the localized mask region supplies the presence determination); and
produce the one or more artifact images comprising the one or more artifacts represented by the one or more localized artifact masks as an output (Li in ¶87-90 discloses retaining a successfully synthesized composite. The proposed combination outputs the generated shadow image following the presence determination).”
Li is analogous art because Li, like Zhang in view of Pugh, is in the field of computational image synthesis and is reasonably pertinent to the problem of determining whether a synthesized image contains the intended inserted content.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify output processing of Zhang in view of Pugh with the synthesized result validation/verification and positive result retention of Li because Li teaches that examining the synthesized result provides a determination that the intended content has been sufficiently incorporated (¶¶87-90), and one of ordinary skill would have recognized that incorporating this feature into the method of Zhang in view of Pugh would identify and retain generated images containing the intended shadow at the localized mask region, with a reasonable expectation of success because the generator produces an image and expected mask region suitable for image based shadow detection without altering the synthesis mechanism.
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Claim 18
Regarding Claim 18, Zhang in view of Pugh teaches “The non-transitory computer-readable storage medium as claimed in claim 16”.
Zhang in view of Pugh does not explicitly teach all of “… wherein the method further comprises: determining the at least one localization parameters based on a size and a dimension of the ROI.”
However, Li teaches “… wherein the method further comprises: determining the at least one localization parameters based on a size and a dimension of the ROI (Li in ¶¶n0081-n0091 discloses dimensionally comparing/fitting an inserted clothing instance to the receiving arm region, defined as its minimum bounding rectangle; ¶¶n0061-n0068 discloses destination relative scale calculations).”
Li is analogous art because Li, like Zhang in view of Pugh, is in the field of mask based synthetic image generation and is reasonably pertinent to the problem of adapting the size and placement of inserted image content to dimensions of a receiving region.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the localization parameter determination for transforming artifact masks of Zhang in view of Pugh with the receiving region dependent scaling and dimension based placement adjustment of Li because Li teaches that dimension dependent scaling and placement provides appropriately fitted image content and reduces gaps at insertion boundaries (¶n0110), and one of ordinary skill would have recognized that incorporating this feature into the method of Zhang in view of Pugh would determine the size and dimensions of the ROI mask and adjust the predefined scaling and placement parameters of the artifact masks according to those dimensions, with a reasonable expectation of success because Zhang already applies geometric transformations to artifact masks, Pugh supplies the image aligned receiving region masks, and Li teaches destination relative scaling, transformation, and dimension based placement adjustments.
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Claims 4 and 15 are rejected under 35 U.S.C. 103 as obvious over Zhang et al (Zhang, X., Barron, J. T., Tsai, Y. T., Pandey, R., Zhang, X., Ng, R., & Jacobs, D. E. (2020). Portrait shadow manipulation. ACM Transactions on Graphics (TOG), 39(4), 78-1, hereafter referred to as Zhang and Pugh et al (US 20210142497 A1, hereafter referred to as Pugh), further in view of Tyagi et al (US10860836B1, hereafter referred to as Tyagi).
Claim 4
Regarding Claim 4, Zhang in view of Pugh teaches “The method as claimed in claim 1”.
Zhang in view of Pugh does not explicitly teach all of “… further comprising: providing feedback for varying one of the plurality of localization parameters and one or more illumination parameters in response to determining that the one or more artifacts represented by the one or more localized artifact masks is absent in the one or more artifact images.”
However, Tyagi teaches “… further comprising: providing feedback for varying one of the plurality of localization parameters (Tyagi in ¶¶14-18, 25-29 discloses detector result feedback to theta learner which determines spatial transformations including translation and scaling) and one or more illumination parameters (Tyagi in ¶25 discloses the same controller determines color scaling that changes brightness and/or intensity) in response to determining that the one or more artifacts represented by the one or more localized artifact masks is absent in the one or more artifact images (Tyagi in ¶¶55-61 discloses failure to detect the inserted target and provision of a hard training indication to the controller in response).”
Tyagi is analogous art because Tyagi, like Zhang in view of Pugh, is in the field of computer generated composite imagery and is reasonably pertinent to the problem of adapting synthetic image generation to detection failures in the resulting images.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the shadow synthesis parameter selection of Zhang in view of Pugh with the detector failure feedback to a controller selecting spatial and brightness/intensity operations of Tyagi because Tyagi teaches that this feedback guided generation of hard positive images provides training examples addressing detector blind spots and improving detector performance (¶¶18, 55-61), and one of ordinary skill would have recognized that incorporating this feature into the method of Zhang in view of Pugh would use a failure to detect the generated shadow to provide feedback for selecting both a subsequent shadow mask localization parameter and a shadow illumination parameter, with a reasonable expectation of success because Zhang already exposes mask-geometry and photometric controls for its synthetic shadows, Pugh teaches trained network shadow detection, and Tyagi provides the image to detector to controller workflow and both categories of controllable transformations.
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Claim 15
Regarding Claim 15, Zhang in view of Pugh teaches “The electronic device as claimed in claim 12”.
Zhang in view of Pugh does not explicitly teach all of “… wherein the processor is further configured to: provide feedback for varying one of the plurality of localization parameters and one or more illumination parameters in response to determining that the one or more artifacts represented by the one or more localized artifact masks is absent in the one or more artifact images.”
However, Tyagi teaches “… wherein the processor is further configured to: provide feedback for varying one of the plurality of localization parameters (Tyagi in ¶¶14-18, 25-29 discloses detector result feedback to theta learner which determines spatial transformations including translation and scaling) and one or more illumination parameters (Tyagi in ¶25 discloses the same controller determines color scaling that changes brightness and/or intensity) in response to determining that the one or more artifacts represented by the one or more localized artifact masks is absent in the one or more artifact images (Tyagi in ¶¶55-61 discloses failure to detect the inserted target and provision of a hard training indication to the controller in response).”
Tyagi is analogous art because Tyagi, like Zhang in view of Pugh, is in the field of computer generated composite imagery and is reasonably pertinent to the problem of adapting synthetic image generation to detection failures in the resulting images.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the shadow synthesis parameter selection of Zhang in view of Pugh with the detector failure feedback to a controller selecting spatial and brightness/intensity operations of Tyagi because Tyagi teaches that this feedback guided generation of hard positive images provides training examples addressing detector blind spots and improving detector performance (¶¶18, 55-61), and one of ordinary skill would have recognized that incorporating this feature into the method of Zhang in view of Pugh would use a failure to detect the generated shadow to provide feedback for selecting both a subsequent shadow mask localization parameter and a shadow illumination parameter, with a reasonable expectation of success because Zhang already exposes mask-geometry and photometric controls for its synthetic shadows, Pugh teaches trained network shadow detection, and Tyagi provides the image to detector to controller workflow and both categories of controllable transformations.
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JUSTIN P CASCAIS whose telephone number is (703)756-5576. The examiner can normally be reached Monday-Friday 8:00-4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mr. O’Neal Mistry can be reached on (313) 446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.P.C./Examiner, Art Unit 2674
/ONEAL R MISTRY/Supervisory Patent Examiner, Art Unit 2674
Date: 9/22/2026
