DETAILED ACTION
Status
This Office Action is responsive to claims filed on 04/28/2015. Please note Claims 1-17 are pending and have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-17 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
Claim 1 recites “the multiscale feature maps of the source images” which lack antecedent basis. The claim defines “a series of multiscale feature maps” of “each source image”. It’s unclear whether “the multiscale feature maps of the source images” refers to a collection (or a subset) of these series of multiscale feature maps. Claim 17 recites similar features of claim 1, and is therefore rejected. (Notice that, claim 9 recites “all of the final encoded feature maps of all of the source images” and “one or more intermediate encoded feature maps of one or more source images”, which are clear.)
Claims 2-8 are dependent from claim 1, and are therefore rejected.
Claim 9 recites “the features” of all of the final encoded feature maps which lacks antecedent basis. The claim fails to mention or define any features of the final encoded feature maps before referring to such limitation. Clarification is needed.
Claims 10-16 are dependent from claim 9, and are therefore rejected.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 4, 6-9, 13-15 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over TANG (US 20220335600 A1), in view of Joachim (US 20240378832 A1).
Regarding Claim 1, TANG discloses a method comprising:
accessing source images ([0005] “The method includes inputting an image into a split-attention network…”);
encoding each source image into a series of multiscale feature maps ([0005] “The method includes inputting an image into a split-attention network to extract a feature map at each scale of multiple scales and compressing the feature map of each scale along a channel dimension to form a compressed feature map of each scale, by an image encoder”);
decoding the target view into a target image of the scene, wherein the decoding involves ([0005] “concatenating, by the decoder, an attention enhanced feature map at a current scale of the multiple scales, in combination with one or more up-sampled feature maps and/or one or more down-sampled feature maps from other scales of the multiple scales, to form a concatenated feature map of the current scale”):
applying global attention across a set of higher-level features of the multiscale feature maps of the source images ([0034] “Then the current scale feature map may be concatenated with all up-sampled and down-sampled feature maps from other scales in a channel dimension, indicating that each concatenated feature map can represent the global contextual and local detail information of the lesion.” [0043] “For example, for the highest-level scale SA (e.g., the uppermost or last SA in FIG. 1), the attention enhanced feature map at the highest-level scale may be concatenated, by the decoder, in combination with 4 up-sampled feature maps by up-sampling the concatenated feature maps outputted from other 4 SAs to form the concatenated feature map of the highest-level scale.”); and
applying local attention across limited sets of lower-level features of the multiscale feature maps of the source images ([0034] “Then the current scale feature map may be concatenated with all up-sampled and down-sampled feature maps from other scales in a channel dimension, indicating that each concatenated feature map can represent the global contextual and local detail information of the lesion.” [0043] “For another example, for the lowest-level scale SA (e.g., the lowermost or first SA in FIG. 1), the attention enhanced feature map at the lowest-level scale may be concatenated, by the decoder, in combination with 4 down-sampled feature maps by down-sampling the attention enhanced feature maps at 4 other scales to form the concatenated feature map of the lowest-level scale.”).
TANG does not expressly disclose source images of a scene and defining a target view for the scene.
However, in the same field of endeavor, Joachim discloses accessing source images of a scene ([0132] “As shown in FIG. 2, the digital image 206 portrays a static, two-dimensional image. In particular, the digital image 206 portrays a two-dimensional projection of a scene that was captured from the perspective of a camera.” [0189] “Specifically, the cascaded modulation inpainting neural network 502 starts with an encoder E that takes the partial image and the mask as inputs to produce multi-scale feature maps from input resolution to resolution 4×4”) and defining a target view for the scene ([0105] “Thus, upon receiving one or more user inputs targeting an object of the digital image for an object-aware modification”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified the method of TANG with the feature of accessing source images of a scene and defining a target view for the scene. Doing so could provide “improved flexibility” to a user, as taught by Joachim.
Regarding Claim 4, TANG-Joachim discloses the method of claim 1, wherein the decoding comprises applying a convolutional layer between the global attention and the local attention (TANG [0006] “a convolutional layer is used to compress the feature map of each scale along the channel dimension to form the compressed feature map of each scale.”).
Regarding Claim 6, TANG-Joachim discloses the method of claim 1, wherein the source images comprise one or more images captured by a mobile device (Joachim [0130] “In one or more embodiments, the scene-based image editing system 106 provides the digital image 206 for display after the digital image 206 is captured via a camera of the client device 204.”).
Regarding Claim 7, TANG-Joachim discloses the method of claim 1, wherein the scene comprises an outdoor scene depicting at least one building (see Joachim Fig. 73. [0131] “In some embodiments, an object comprises an instance of an identifiable thing (e.g., a person, an animal, a building, a car, or a cloud, clothing, or some other accessory).”).
Regarding Claim 8, TANG-Joachim discloses the method of claim 1, wherein the scene comprises an interior scene depicting at least one object (see Joachim Fig. 68A. [0131] “In some embodiments, an object comprises an instance of an identifiable thing (e.g., a person, an animal, a building, a car, or a cloud, clothing, or some other accessory).”).
Regarding Claim 9, it recites similar limitations of claim 1. The rationale of claim 1 rejection is applied to reject claim 9. Notice that claim 9 recites the additional features of a final encoded feature map and a depth map feature projection process, which are taught by Joachim in [0183]-[0184] and [0303], respectively.
Regarding Claim 13, it recites similar limitations of claim 4. The rationale of claim 4 rejection is applied to reject claim 13.
Regarding Claim 14, TANG-Joachim discloses the method of claim 13, wherein the decoding comprises applying additional sets of alternating convolutional layers and local attention layers (TANG [0032] “in order to relieve the computation burden, the multiple scale feature maps may be compressed to 32 channels using a convolutional layer with 32 3×3 kernels.”).
Regarding Claim 15, TANG-Joachim discloses the method of claim 9, wherein: the decoding involves decoding the representation of the target view into a series of multiscale feature maps including a series of intermediate decoded feature maps followed by a final decoded feature map (Joachim [0184] “As illustrated, in one or more embodiments, the cascaded modulation inpainting neural network 502 generates a global feature code from the final encoded feature vector of the encoder 504.”); and the decoding involves applying a final activation function to classify the features of the final decoded feature map into image pixels (Joachim [0467] “In one or more embodiments, a heatmap (also referred to as a class activation map) includes a prediction made by a convolutional neural network that indicates a probability value, on a scale of zero to one, that a specific pixel of an image belongs to a particular class from a set of classes.”).
Regarding Claim 17, it recites similar limitations of claim 1. The rationale of claim 1 rejection is applied to reject claim 17.
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over TANG (US 20220335600 A1), in view of Joachim (US 20240378832 A1), further in view of ZHOU (US 20240242365 A1).
Regarding Claim 2, TANG-Joachim discloses the method of claim 1. In the same field of endeavor, ZHOU discloses a method further comprising performing a depth map feature projection process to determine a limited set of lower-level features for the local attention ([0006] “In a general aspect, here is provided a processor-implemented method including generating a depth-aware feature of an image dependent on image features extracted from image data of the image and generating image data, representing information corresponding to one or more segmentations of the image, based on the depth-aware feature and a depth-aware representation” [0008] “The generating of the depth-aware feature by fusing the visual feature and the depth feature may include generating a first visual feature and a first depth feature by performing a convolution operation on the visual feature and the depth feature, respectively, generating a first feature by fusing the first visual feature and the first depth feature”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified the method of TANG-Joachim with the feature of performing a depth map feature projection process to determine a limited set of lower-level features. Doing so could “improve the algorithm accuracy”, as taught by ZHOU.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over TANG (US 20220335600 A1), in view of Joachim (US 20240378832 A1), further in view of WANG (US 20240193923 A1).
Regarding Claim 5, TANG-Joachim discloses the method of claim 1. In the same field of endeavor, Wang discloses wherein the source images comprise one or more aerial images of the scene (WANG Claim 11 “A method of detecting a target object, the method comprising: by using a target object detection model, extracting a plurality of feature maps of an image to be detected…” and Claim 12 “…the image to be detected is an image captured by an unmanned aerial vehicle”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified the method of TANG-Joachim with the feature of comprising one or more aerial images of the scene in the source images. The type of source images to be processed is entirely a designer’s choice. The system disclosed in TANG, Joachim, and WANG can process any type of digital images.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHONG WU whose telephone number is (571)270-5207. The examiner can normally be reached MON-FRI: 9AM-5PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao Wu can be reached at 571-272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHONG WU/Primary Examiner, Art Unit 2613