Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception without significantly more. Claim 1 is directed to an apparatus including at least one memory and at least one processor, and therefore falls within a statutory category as a machine. Claims 2–20 depend directly or indirectly from claim 1 and likewise fall within the statutory category of a machine. However, claim 1 recites an abstract idea, such as a mathematical concept. The claim recites a mathematical optimization process by receiving an input, representing the input as a graph having nodes and edges, processing a subset of the input based on the graph cut, and generating a prediction, which are data-processing steps associated with the mathematical concept. The claim(s) recite(s) a processor, memory, and ANN which are used as generic tools to perform the abstract mathematical/data-processing operations. This judicial exception is not integrated into a practical application because the additional elements do not amount to significantly more than an abstract idea. Claims 2–20 do not cure the deficiency of claim 1. Claims 2–20 further limit the abstract graph-based ANN processing and do not recite additional elements that integrate the abstract idea into a practical application. The dependent claims merely further define the data, graph representation, ANN processing, mathematical optimization, graph cut, or prediction generation, and therefore remain directed to the abstract idea. Accordingly, claims 1–20 are directed to an abstract idea without significantly more.
Claim Rejections - 35 USC 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over D1 [Boykov, Yuri, and Gareth Funka-Lea. “Graph Cuts and Efficient N-D Image Segmentation.”] in view of D2 [Jiang, Jindong, and Sungjin Ahn. “Generative Neurosymbolic Machines.” ].
Claim 1: An apparatus, comprising: at least one memory; and at least one processor coupled to the at least one memory, the at least one processor configured to: D1 teaches that the graph-cut/image-segmentation process is performed using a computer-based implementation, which would include at least one processor coupled to memory to execute the disclosed segmentation algorithm. See D1, p. 123, section 3; p. 111, section 1.1.
receive, via an D1 teaches receiving an input image to be segmented and iterative parameter re-estimation and learning. See D1, Abstract.
represent the input as a graph, the graph including a plurality of nodes connected by edges; D1 teaches representing the input image as a graph, where the pixels correspond to nodes and the links between pixels correspond to edges in the graph. See D1, Fig. 3; p. 116, section 2.1.
determine, D1 teaches determining a graph cut between source and sink nodes associated with the input image, where the graph cut corresponds to image segmentation. See D1, Fig. 3; p. 116, section 2.1. D1 further teaches solving the graph-based segmentation problem using a constrained optimization process. See D1, pp. 115–116, section 2.
process, D1 teaches that the graph cut produces an image segment, which is a subset of the input image based on the graph cut. See D1, Fig. 3; p. 116, section 2.1.
D1 does not explicitly teach receiving the input via an artificial neural network; however, D2 teaches using neural networks, including graph-based neural networks, for graph-based image processing - see D2, p. 4, section 3.2. D1 does not explicitly teach determining the graph cut via an ANN, D2 teaches applying neural networks, including graph-based neural networks, to graph-based image processing - see D2, p. 4, section 3.2. D1 does not explicitly teach processing the subset via an ANN to generate a prediction, D2 teaches neural-network-based processing of graph-structured image data to generate graph-based image-processing outputs - see D2, p. 4, section 3.2. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the graph-cut image segmentation system of D1 to use the ANN/graph neural network processing taught by D2 because D1 already represents an input image as graph-structured data for segmentation, and D2 teaches that neural networks are suitable for graph-based image processing. The combination would have amounted to applying a known neural-network-based graph processing technique to a known graph-cut image segmentation process to obtain predictable results, namely ANN-based processing of graph nodes and edges to generate segmentation or prediction output.
Claim 2. The apparatus of claim 1, in which the graph cut segments the input such that a weight of cross edges between segments is smallest. D1, p. 116, section 2.1 teaches “Our goal is to compute the best cut that would give an ‘optimal’ segmentation. In combinatorial optimization the cost of a cut is defined as the sum of the costs of edges that it severs”; p. 117 “the minimum cost cut C on graph G can be computed exactly in polynomial time”. D1 further teaches that the cost of the cut |C| is computed as the sum of the weights of severed edges. See D1, p. 117 “Note that severed n-links are located at the segmentation boundary. Thus, their total cost represents the cost of segmentation boundary.”.
Claim 3. The apparatus of claim 1, in which the input is an image or a grid of features. D2, p. 3, section 3.1 teaches “we first build from the global representation z^d a feature map f of (H × W × d_f)-dimension with H and W being the spatial dimension and d_f being the feature dimension in each spatial position”; p. 4, section 3.2 “StructDRAW draws not pixels but an abstract structure on feature space, i.e., the latent feature map”. D2’s feature map is a spatial grid where each position contains a feature vector. See D2, p. 3, section 3.1, Eq. (2).
Claim 4. The apparatus of claim 3, in which the plurality of nodes corresponds to pixels of the image or the grid of features. D1 teaches teaches that the plurality of nodes corresponds to pixels of the image. See D1, p. 115, section 2.1 (“The nodes of our graphs represent image pixels or voxels.”); p. 116, Fig. 3(a) (“Image pixels form a 2D grid graph”); p. 121, section 2.5 (“To segment a given image we create a graph G = (V, ε) with nodes corresponding to pixels p ∈ P of the image.”). D2 teaches representing inputs as feature grids where each spatial position (h, w) in the grid corresponds to a feature vector. See D2, p. 3, section 3.1. When the graph-cut method of D1 is applied to a grid of features as taught by D2, the plurality of nodes would naturally correspond to the spatial positions in the feature grid, just as nodes in D1 correspond to pixel positions in the image grid. This is a direct and predictable application of the graph construction principle taught by D1 to the feature-grid representation taught by D2.
Claim 5. The apparatus of claim 3, in which the grid of features corresponds to image features of a scene including one or more objects in the image, and the at least one processor is further configured to divide the image features of the scene among the one or more objects. See D1, p. 117, section 2.2 (describing how the cut partitions pixels into object and background according to Eq. (10)).
Claim 6. The apparatus of claim 1, in which the graph cut is differentiable. See D1, p. 117, section 2.2 (describing how the cut partitions pixels into object and background according to Eq. (10)). The object and background is differentiable.
Claim 7. The apparatus of claim 1, in which the ANN includes one or more differentiable optimization layers. See D2, p. 4, section 3.4 (“Learning”) (“We train the model by optimizing the following Evidence Lower Bound (ELBO)”); Eq. (4) (showing optimization objective).
Claims 8-14 are rejected for similar reasons as to those described in claims 1-7 respectively.
Claims 15-20 are rejected for similar reasons as to those described in claims 1-6 respectively.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Amandeep Saini whose telephone number is (571)272-3382. The examiner can normally be reached M-F (8AM-4PM).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AMANDEEP SAINI/Supervisory Patent Examiner, Art Unit 2662
Prosecution Insights