DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 10/10/2024 submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “a transparentize unit configured to…”, “a synthesis unit configured to…”, “a deletion unit configured to…”, a data generation unit configured to…” in claims 1-4.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof. The original specifications in fig. 5 , device 10 discloses processor hardware or their equivalents thereof to execute the functions.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (CN 113056905 A) in view of Dhua et al. (US Patent No. 10049308 B1).
Regarding Claim 1,
Zhang discloses A pseudo image generation device that generates a pseudo image from foreground image data pertaining to a foreground image in which an object to be synthesized is reflected and background image data pertaining to a background image in which a background is reflected, the pseudo image generation device comprising: a transparentizing processing unit configured to transparentize a region other than the object to be synthesized in the foreground image; (Zhang, Specific implementation examples, discloses the processor 220 may generate a background mask 424 for image 420. For example, the processor 220 may generate a depth of view for the image 420, and use the same method as the separation foreground and background introduced in step 1, the processor 220 may decompose the image 420 to obtain a binarization background mask 424. Different from the foreground mask shown in FIG. 4 A, the background region of the background mask 424 may be white or transparent, and the foreground region of the background mask 424 may be black. In some embodiments, the background mask 424 may be an alpha hybrid mask. By mixing the background mask 424 with a significant FIG. 422, the processor 220 may obtain a modified significant FIG. 426 having only a background significance. In FIG. 4 G, the obvious graph after modification shows the outline characteristics of the background building near the riverbank (as shown in circle); region other than subject (object) in a foreground image is altered to make it white (transparent) or its brightness values are changed to make it transparent when editing and blending images for aesthetic or improve visibility)
Zhang does not explicitly disclose a synthesis unit configured to synthesize the background image data and the foreground image data after transparentizing processing such that the foreground image after the transparentizing processing in the transparentizing processing unit is superimposed on a prescribed position in the background image; a deletion unit configured to delete background annotation data overlapping the object to be synthesized after synthesis in the synthesis unit from the background annotation data included in the background image data and attached within the background image; a data generation unit configured to generate annotation data of the pseudo image based on the background annotation data after deletion in the deletion unit.
Dhua discloses a synthesis unit configured to synthesize the background image data and the foreground image data after transparentizing processing such that the foreground image after the transparentizing processing in the transparentizing processing unit is superimposed on a prescribed position in the background image; (Dhua, Col. 5, Lines 47-60, discloses consider the background image 300 illustrated in FIG. 3A. There are various shades, textures, and shapes that are part of the background. If the item portion 322, including the non-item regions 324, is layered onto or pasted into the background image to generate a synthesized image 320, as illustrated in the example of FIG. 3B, the neural network may determine that the non-item region is part of the item itself. This may particularly be the case if several images are used for training that all demonstrate such an artifact. If a query image is subsequently received that does not contain such an artifact then the neural network may not properly classify the item; object from image is synthesized at a specific position with background image)
a deletion unit configured to delete background annotation data overlapping the object to be synthesized after synthesis in the synthesis unit from the background annotation data included in the background image data and attached within the background image; (Dhua, Col. 8, Lines 14-56, discloses (30) Accordingly, additional and/or alternative steps can be used to attempt to remove or at least further reduce the presence of such artifacts in the synthesized images. In various embodiments, one or more morphological operations can be performed on the mask to attempt to remove the remaining background pixels. The morphological operations can include operations such as opening, erosion, and/or dilation, among other such options. The operations can also include combinations or sequences of these operations. For example, an approach in accordance with one embodiment utilizes an erosion followed by an opening operation, where the opening involves both an erosion and a dilation. In this example, the initial erosion is performed using a slightly larger structuring element than is used for the opening operation. The erosion process involves removing pixels from around an edge using a structuring element or tool that makes the object thinner and can in at least some embodiments also smooth the edge based at least in part upon the size and shape of the tool. The second erosion also makes the object smaller by taking away some of the edge information, followed by a dilation that adds some thickness back, such as by blending or extending the pixel values near the edge. Both morphological operations can be performed using structuring elements (e.g., circles) that are a fraction of the size of the overall mask dimensions. As mentioned, the second erosion and dilation can utilize a smaller structuring element in order to generate a finer edge that would otherwise result from the first erosion, but the first erosion will remove a significantly larger portion of the artifact region. In embodiments that utilize alpha blending, the morphological operation(s) can be performed before the alpha blending, such that the artifact can be substantially removed before alpha blending is performed to remove any resulting jagged edge regions due to the background mask. While the removal of shadow regions and intra -item regions may not be perfect, these artifacts will differ appreciably between images such that there should be no significant impact on the overall training of the model. The impact has been verified through experimentation. The morphological processing discussed herein can generate a mask that enables the item image region to be blended into the background image with minimal detectable artifacts that would impact the network training; artifact objects or intra region objects that are noise (annotation data) are removed from background image when synthesized with object image during process of alpha blending image) and
a data generation unit configured to generate annotation data of the pseudo image based on the background annotation data after deletion in the deletion unit. (Dhua, Col. 6, Lines 57-67, Col. 7, Lines 1-35, discloses background images are selected from a set of possible background images based on one or more criteria. These criteria can include, for example, that the image includes an indoor or outdoor scene, and not a representation of another object, as that may create confusion as to which item(s) or object(s) in the image correspond to the object for training. Further, scene images that include other persons or items (i.e., apparel items for apparel training) may be excluded as well for similar reasons. There may be other criteria as well, such as minimum or maximum size or resolution, brightness, contrast, and the like. For images with annotations or metadata indicating what is represented, this data can be used to select and/or exclude background images for consideration. In some embodiments, the images can also be processed using face detectors, object recognition algorithms, or other such approaches to attempt to determine background images that include representations of persons or objects that should cause those images to be excluded from consideration. In some embodiments the selection can be further based on the ability to locate an affordance region in the image. An “affordance” region as utilized herein refers to a location where a person might typically be represented in an image, such as may involve standing on a floor, ground, grass, sidewalk, path, road, land, sand, snow, carpet, runway, or field, among other such options. While an image showing a sky, sea, or galaxy might qualify as a background image, images including persons in front of those backgrounds are very unlikely and it may be preferable to utilize backgrounds that are similar to the types of backgrounds that will actually be encountered in real world query images. Further, aspects such as scale and location can be used in some embodiments such that item images including an entire person might be selected for backgrounds including a floor to ceiling view, while item images of just a shirt or shorts might include a representation of a much smaller region, among other such options. In some embodiments images can be analyzed to attempt to remove substantially redundant images, to avoid training on features of that type of background. Similarly, sub-regions of the various background images may be selected in at least some embodiments to further differentiate the backgrounds in the various synthesized images, such as where a single background might be used for ten different item images, but different regions of that background selected for each synthesized image. The selection can be performed randomly in some embodiments, or according to a selection algorithm in others; object image data is annotated with relevant information pertaining to the object and background image)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Zhang in view of Dhua having a method of changing region other than subject with clear white or bright background (transparentize) with the teachings of Dhua having, by the training module synthesizing an object and background image where any background noise or intra media on superposed on the background image is deleted in order to improve aesthetic value or visibility in an image.
Regarding Claim 2,
The combination of Zhang and Dhua further discloses wherein the deletion unit further converts foreground annotation data included in the foreground image data and attached within the foreground image according to a position where the foreground image is superimposed on the background image, and the data generation unit generates the annotation data of the pseudo image based on the foreground annotation data after conversion and the background annotation data after deletion. (Dhua, Col. 6, Lines 57-67, Col. 7, Lines 1-35, discloses background images are selected from a set of possible background images based on one or more criteria. These criteria can include, for example, that the image includes an indoor or outdoor scene, and not a representation of another object, as that may create confusion as to which item(s) or object(s) in the image correspond to the object for training. Further, scene images that include other persons or items (i.e., apparel items for apparel training) may be excluded as well for similar reasons. There may be other criteria as well, such as minimum or maximum size or resolution, brightness, contrast, and the like. For images with annotations or metadata indicating what is represented, this data can be used to select and/or exclude background images for consideration. In some embodiments, the images can also be processed using face detectors, object recognition algorithms, or other such approaches to attempt to determine background images that include representations of persons or objects that should cause those images to be excluded from consideration. In some embodiments the selection can be further based on the ability to locate an affordance region in the image. An “affordance” region as utilized herein refers to a location where a person might typically be represented in an image, such as may involve standing on a floor, ground, grass, sidewalk, path, road, land, sand, snow, carpet, runway, or field, among other such options. While an image showing a sky, sea, or galaxy might qualify as a background image, images including persons in front of those backgrounds are very unlikely and it may be preferable to utilize backgrounds that are similar to the types of backgrounds that will actually be encountered in real world query images. Further, aspects such as scale and location can be used in some embodiments such that item images including an entire person might be selected for backgrounds including a floor to ceiling view, while item images of just a shirt or shorts might include a representation of a much smaller region, among other such options. In some embodiments images can be analyzed to attempt to remove substantially redundant images, to avoid training on features of that type of background. Similarly, sub-regions of the various background images may be selected in at least some embodiments to further differentiate the backgrounds in the various synthesized images, such as where a single background might be used for ten different item images, but different regions of that background selected for each synthesized image. The selection can be performed randomly in some embodiments, or according to a selection algorithm in others; object image data is annotated with relevant information pertaining to the object and background image). Additionally, the rational and motivation to combine the reference Zhang and Gupta as applied in rejection of claim 1 apply to this claim.
Regarding Claim 3,
The combination of Zhang and Dhua further discloses wherein the transparentizing processing unit is configured to cut a target region including the object to be synthesized in the foreground image from the foreground image and transparentize a region other than the object to be synthesized in the target region obtained by cutting. (Zhang, Specific implementation examples, discloses the processor 220 may determine the image 420 on the second cutting area. For example, the second cutting area may be a rectangle with four boundaries. To this end, the processor 220 may determine the background tailoring frame 427 that satisfies one or more of the following criteria on the background binary map 424: (1) the background tailoring frame may include a background object corresponding to all or most of the significance of the significance; (2) the length-width ratio of the background cutting frame can be the same as the length-width ratio of the original image 310; (3) the background cutting frame can be connected with at least one geometric marking point (A, B, C and/or D) of the target object (i.e., using at least one geometric mark point coordinate determining cutting area); and (4) the foreground main object in the background cutting frame 427 can not be more than the main object of the first cutting area 320. For example, because the background cutting frame 427 can be used for determining the second cutting area, the second cutting area is then mixed with the first cutting area, so for all possible position of the background cutting frame; The method described herein may select a smaller portion of the primary object to avoid potential defects and/or problems in the mixing process. For example, in FIG. 4G, the background cutting frame 427 comprises a background building, and the left side point B of the face on the background cutting frame 427 of the right boundary; subject or object region is cut and synthesized with background image). Additionally, the rational and motivation to combine the reference Zhang and Gupta as applied in rejection of claim 1 apply to this claim.
Regarding Claim 4,
The combination of Zhang and Dhua further discloses wherein the transparentizing processing unit is configured to cut a target region including the object to be synthesized in the foreground image from the foreground image and transparentize a region other than the object to be synthesized in the target region obtained by cutting. (Zhang, Specific implementation examples, discloses the processor 220 may determine the image 420 on the second cutting area. For example, the second cutting area may be a rectangle with four boundaries. To this end, the processor 220 may determine the background tailoring frame 427 that satisfies one or more of the following criteria on the background binary map 424: (1) the background tailoring frame may include a background object corresponding to all or most of the significance of the significance; (2) the length-width ratio of the background cutting frame can be the same as the length-width ratio of the original image 310; (3) the background cutting frame can be connected with at least one geometric marking point (A, B, C and/or D) of the target object (i.e., using at least one geometric mark point coordinate determining cutting area); and (4) the foreground main object in the background cutting frame 427 can not be more than the main object of the first cutting area 320. For example, because the background cutting frame 427 can be used for determining the second cutting area, the second cutting area is then mixed with the first cutting area, so for all possible position of the background cutting frame; The method described herein may select a smaller portion of the primary object to avoid potential defects and/or problems in the mixing process. For example, in FIG. 4G, the background cutting frame 427 comprises a background building, and the left side point B of the face on the background cutting frame 427 of the right boundary; subject or object region is cut and synthesized with background image). Additionally, the rational and motivation to combine the reference Zhang and Gupta as applied in rejection of claim 1 apply to this claim.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
US Pub No. 20200265623-A1 (Gupta et al., This application relates generally to computer-
implemented methods and systems for computer graphics processing. Specifically, the application involves automatically selecting and blending images based on image aesthetic scores. Computer systems are provided that may blend an automatically selected foreground image with an automatically selected background image to produce an editable photo creation, Abstract)
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PINALBEN V PATEL whose telephone number is (571)270-5872. The examiner can normally be reached M-F: 10am - 8pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Wills-Burns Chineyere can be reached at 571-272-9752. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Pinalben Patel/Examiner, Art Unit 2673
Prosecution Insights