DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Examiner Note
Examiner cites particular columns, paragraphs, figures and line numbers in the references as applied to the claims below for the convenience of the applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested that, in preparing responses, the applicant fully consider the references in their entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the examiner. The entire reference is considered to provide disclosure relating to the claimed invention. The claims & only the claims form the metes & bounds of the invention. Office personnel are to give the claims their broadest reasonable interpretation in light of the supporting disclosure. Unclaimed limitations appearing in the specification are not read into the claim. Prior art was referenced using terminology familiar to one of ordinary skill in the art. Such an approach is broad in concept and can be either explicit or implicit in meaning. Examiner's Notes are provided with the cited references to assist the applicant to better understand how the examiner interprets the applied prior art. Such comments are entirely consistent with the intent & spirit of compact prosecution.
Drawings
Figures 1-9, 21-25 should be designated by a legend such as --Prior Art-- because only that which is old is illustrated. See MPEP § 608.02(g). Corrected drawings in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. The replacement sheet(s) should be labeled “Replacement Sheet” in the page header (as per 37 CFR 1.84(c)) so as not to obstruct any portion of the drawing figures. If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
The drawings are objected to because Fig. 26 contains illegible labels for regions 1-9. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
The following is a quotation of pre-AIA 35 U.S.C. 112, fourth paragraph:
Subject to the following paragraph [i.e., the fifth paragraph of pre-AIA 35 U.S.C. 112], a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
Claims 10, 11, and 12 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Use of “and/or” in these claims is being interpreted as “or” according to BRI and for compact prosecution. Additionally, regarding claim 12, all limitations (the “first computer program” and “second computer program”) will be treated for analysis for compact prosecution.
Claim 17 is rejected under 35 U.S.C. 112(d) or pre-AIA 35 U.S.C. 112, 4th paragraph, as being of improper dependent form for failing to further limit the subject matter of the claim upon which it depends, or for failing to include all the limitations of the claim upon which it depends. Claim 17 is a duplicate of parent claim 16 with identical limitations, and therefore does not limit the scope of the claims. Applicant may cancel the claim(s), amend the claim(s) to place the claim(s) in proper dependent form, rewrite the claim(s) in independent form, or present a sufficient showing that the dependent claim(s) complies with the statutory requirements.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 USC 101 as the claimed invention is directed to abstract ideas without significantly more. The claim(s) recite(s) mental processes and mathematical concepts.
Step 1, Statutory Category:
Claims 1-11 are a process. Claims 12-20 are a manufacture.
Step 2A Prong I, Judicial Exception:
The examiner submits that the foregoing claim limitations constitute a mental process, as
the claims cover performance of the human mind, given their broadest reasonable interpretation.
Abstract ideas are bolded.
Claim 1 recites the limitations:
A computer-implemented method for segmenting a building scene
the method comprising: obtaining a training dataset of top-down depth maps,
each depth map comprising labeled line segments and junctions between line segments;
and learning, based on the training dataset, a neural network,
the neural network being configured to take as input a top-down depth map of a building scene including building partitions and to output a scene wireframe including the partitions and junctions between the partitions.
The limitation “and learning, based on the training dataset, a neural network” given its broadest reasonable interpretation constitutes mental process and mathematical algorithms that can be performed mentally. A person can perform the mental process of tracking the state of a network and additionally can perform mathematical concepts such as a backpropagation algorithm or gradient decent algorithm to “learn … a neural network”.
Step 2A Prong II, Integration into a Practical Application:
Claim 1 recites the following additional claim limitations outside the abstract idea which
only present general fields of use, mere instructions to apply an exception, and/or insignificant
extra-solution activity:
A computer-implemented method (mere instructions to apply an exception, see MPEP § 2106.05(f))
for segmenting a building scene (general field of use, see MPEP § 2106.05(h))
the method comprising: obtaining a training dataset of top-down depth maps, (insignificant extra-solution activity, mere data gathering, see MPEP § 2106.05(g))
each depth map comprising labeled line segments and junctions between line segments; (general field of use, see MPEP § 2106.05(h))
the neural network being configured to take as input a top-down depth map of a building scene including building partitions and to output a scene wireframe including the partitions and junctions between the partitions. (general field of use, see MPEP § 2106.05(h))
Step 2B, Significantly More:
When considered individually or in combination, the additional limitations and elements
of claim 1 do not amount to significantly more than the judicial exceptions for the same reasons above as to why the additional limitations do not integrate the abstract idea into a practical application.
The additional limitations identified as mere instructions to apply an exception,
insignificant extra-solution activity, or general field of use above are carried over and also do not
provide significantly more than the abstract idea. See MPEP § 2106.04(d) referencing MPEP §
2106.05(h) and MPEP § 2106.05(g).
Merely obtaining a labeled dataset for training a neural network is considered well understood, routine, and conventional activity in the art.
Considering the claim limitations in combination and the claims as a whole does not
change this conclusion and claim 1 is ineligible under 35 U.S.C 101.
Regarding claim 2, in addition to the abstract idea of claim 1, claim 2 recites an additional limitation:
The computer-implemented method of claim 1, wherein each top-down depth map of the training dataset includes: random points, and line segments each between a respective pair of points, the line segments having random heights. (general field of use, see MPEP § 2106.05(h))
Providing information as to the structure of the training data, which is to only be applied to the judicial exception (abstract idea) of “learning … a neural network”, merely links the judicial exception to a particular field of transforming heightmap data.
When considered individually or in combination, the additional limitations and elements of claim 2 do not amount to significantly more than the judicial exceptions for the same reasons above as to why the additional limitations do not integrate the abstract idea into a practical application.
Considering the claim limitations in combination and the claims as a whole does not
change this conclusion and claim 2 is ineligible under 35 U.S.C 101.
Regarding claim 3, in addition to the abstract idea of claim 1, claim 3 recites an additional limitation:
The computer-implemented method of claim 1, wherein one or more top-down depth maps of the training dataset include one or more distractors, a distractor being any other object than a line segment. (general field of use, see MPEP § 2106.05(h))
Providing information as to the structure of the training data, which is to only be applied to the judicial exception (abstract idea) of “learning … a neural network”, merely links the judicial exception to a particular field of transforming heightmap data.
When considered individually or in combination, the additional limitations and elements of claim 3 do not amount to significantly more than the judicial exceptions for the same reasons above as to why the additional limitations do not integrate the abstract idea into a practical application.
Considering the claim limitations in combination and the claims as a whole does not
change this conclusion and claim 3 is ineligible under 35 U.S.C 101.
Regarding claim 4, in addition to the abstract idea of claim 2, claim 4 recites an additional limitation:
The computer-implemented method of claim 2, wherein one or more of the top-down depth maps include noise. (general field of use, see MPEP § 2106.05(h))
Providing information as to the structure of the training data, which is to only be applied to the judicial exception (abstract idea) of “learning … a neural network”, merely links the judicial exception to a particular field of transforming heightmap data.
When considered individually or in combination, the additional limitations and elements of claim 4 do not amount to significantly more than the judicial exceptions for the same reasons above as to why the additional limitations do not integrate the abstract idea into a practical application.
Considering the claim limitations in combination and the claims as a whole does not
change this conclusion and claim 4 is ineligible under 35 U.S.C 101.
Regarding Claim 5, the claim recites the abstract limitations (bolded):
A method of applying a neural network learnable according to a computer-implemented method
for segmenting a building scene,
the method for segmenting a building scene including obtaining a training dataset of top-down depth maps
each depth map comprising labeled line segments and junctions between line segments,
and learning, based on the training dataset, a neural network,
the neural network being configured to take as input a top-down depth map of a building scene comprising building partitions and to output a scene wireframe including the partitions and junctions between the partitions,
the method of applying comprising: obtaining a top-down depth map of a building scene comprising building partitions;
and applying the neural network to the obtained top-down depth map to obtain a wireframe of the building scene,
the wireframe including the partitions and junctions between the partitions.
The limitations “and learning, based on the training dataset, a neural network” and “and applying the neural network to the obtained top-down depth map” given the broadest reasonable interpretation constitute mental processes and mathematical algorithms that can be performed mentally. A person can perform the mental process of tracking the state of a network, and additionally can perform mathematical concepts such as a backpropagation algorithm and gradient decent algorithm to “learn … a neural network” and a forward propagation algorithm to “apply the neural network” respectively.
Claim 5 recites the following additional claim limitations outside the abstract idea which
only present general field of use, mere instructions to apply an exception, and/or insignificant extra-solution activity:
A method of applying a neural network learnable according to a computer-implemented method (mere instructions to apply an exception, see MPEP § 2106.05(f))
for segmenting a building scene, (general field of use, see MPEP § 2106.05(h))
the method for segmenting a building scene including obtaining a training dataset of top-down depth maps (insignificant extra-solution activity, mere data gathering, see MPEP § 2106.05(g))
each depth map comprising labeled line segments and junctions between line segments, (general field of use, see MPEP § 2106.05(h))
the neural network being configured to take as input a top-down depth map of a building scene comprising building partitions and to output a scene wireframe including the partitions and junctions between the partitions, (general field of use, see MPEP § 2106.05(h))
the method of applying comprising: obtaining a top-down depth map of a building scene comprising building partitions; (insignificant extra-solution activity, mere data gathering, see MPEP § 2106.05(g))
to obtain a wireframe of the building scene, (general field of use, see MPEP § 2106.05(h))
the wireframe including the partitions and junctions between the partitions. (general field of use, see MPEP § 2106.05(h))
When considered individually or in combination, the additional limitations and elements
of claim 5 do not amount to significantly more than the judicial exceptions for the same reasons above as to why the additional limitations do not integrate the abstract idea into a practical application.
The additional limitations identified as mere instructions to apply an exception,
insignificant extra-solution activity, or general field of use above are carried over and also do not
provide significantly more than the abstract idea. See MPEP § 2106.04(d) referencing MPEP §
2106.05(h) and MPEP § 2106.05(g).
A method of applying “a neural network learnable according to a computer-implemented method” constitutes mere instructions to apply a judicial exception (abstract idea) on a computer. Obtaining a labeled dataset for training a neural network is mere data gathering, and further, considered well understood, routine, and conventional activity in the art. Providing information as to the structure of training data and output data, which is to only be applied to the judicial exceptions, merely links the judicial exceptions to a particular field of use.
Considering the claim limitations in combination and the claims as a whole does not
change this conclusion and claim 5 is ineligible under 35 U.S.C 101.
Regarding claim 6, in addition to the limitations of claim 5, claim 6 recites the limitations:
The method of claim 5, further comprising computing 2D regions of the obtained depth map
by extracting, from the obtained wireframe, 2D regions of which contours are formed by line segments of the obtained wireframe.
The limitations constitute an additional abstract idea (mental process). A person can identify 2d regions based on a wireframe. This would constitute mental observation and judgement.
These limitations have been considered in combination with the limitations required by the claim(s) from which this claim depends. The additional limitations are considered to constitute additional mental processes under step 2A prong I of the abstract idea analysis, see MPEP § 2106.04(a)(2)(III). The additional limitations and/or additional elements do not integrate the claim limitations into a practical application (step 2A prong II), or recite significantly more than the abstract idea (step 2B). Therefore, claim 6 is ineligible under 35 U.S.C 101.
Regarding claim 7, in addition to the limitations of claim 6, claim 7 recites the limitations:
The method of claim 6,
wherein computing the 2D regions includes: obtaining a graph
having: graph nodes each representing a junction, and graph edges each representing a line segment between two junctions represented two graph nodes, each graph edge comprising two half-edges having opposite orientations;
and determining, using the half-edges: regions of the graph delimited by graph edges and not crossed by any graph edge, and overall edge contours of the graph.
The limitation “and determining, using the half-edges: regions of the graph delimited by graph edges and not crossed by any graph edge, and overall edge contours of the graph” constitutes additional mental process. A person can make observations and judgements about a graph and perform graph operations as a mental process.
Claim 7 recites the following additional claim limitations outside the abstract idea which
only present general fields of use, mere instructions to apply an exception, and/or insignificant extra-solution activity:
wherein computing the 2D regions includes: obtaining a graph (insignificant extra-solution activity, mere data gathering, see MPEP § 2106.05(g))
having: graph nodes each representing a junction, and graph edges each representing a line segment between two junctions represented two graph nodes, each graph edge comprising two half-edges having opposite orientations; (general field of use, see MPEP § 2106.05(h))
When considered individually or in combination, the additional limitations and elements
of claim 7 do not amount to significantly more than the judicial exceptions for the same reasons above as to why the additional limitations do not integrate the abstract idea into a practical application.
Providing information as to the structure of a graph, which is to only be applied to the judicial exceptions, merely links the judicial exceptions to a particular field of use.
Considering the claim limitations in combination and the claims as a whole does not
change this conclusion and claim 5 is ineligible under 35 U.S.C 101.
Regarding claim 8, in addition to the limitations of claim 7, claim 8 recites the limitations:
The method of claim 7,
wherein computing the 2D regions further includes: computing, using a Shoelace formula, areas of each determined region of the graph and overall edge contour of the graph;
and discarding, using the computed areas: regions having a negative area, regions having an area lower than a predefined threshold, and regions having a width lower than a predefined threshold.
The limitations “wherein computing the 2D regions further includes: computing, using a Shoelace formula, areas of each determined region of the graph and overall edge contour of the graph;” and “discarding, using the computed areas: regions having a negative area, regions having an area lower than a predefined threshold, and regions having a width lower than a predefined threshold” constitute abstract ideas, specifically mental processes and mathematical equations. A person can perform the mental process of performing mathematical equations, such as the “Shoelace formula”. Additionally, a person can make the mental judgement to discard specific areas based on width.
These limitations have been considered in combination with the limitations required by the claim(s) from which this claim depends. The additional limitations are considered to constitute additional mental processes under step 2A prong I of the abstract idea analysis, see MPEP § 2106.04(a)(2)(III). The additional limitations and/or additional elements do not integrate the claim limitations into a practical application (step 2A prong II), or recite significantly more than the abstract idea (step 2B). Therefore, claim 8 is ineligible under 35 U.S.C 101.
Regarding claim 9, in addition to the limitations of claim 6, claim 9 recites the limitations:
The method of claim 6,
wherein the obtained depth map stems from a 3D point cloud,
and the method further comprises: projecting the computed 2D regions on the 3D point cloud, thereby obtaining a 3D segmentation of the building scene.
The limitation “projecting the computed 2D regions on the 3D point cloud, thereby obtaining a 3D segmentation of the building scene” constitutes an abstract idea (mental process). A person can perform the mental process of computing a projection from 2 dimensions into a 3 dimensional point cloud, for example, by performing matrix multiplication with the region data and projection matrices.
Claim 9 recites the following additional claim limitations outside the abstract idea which
only present general fields of use, mere instructions to apply an exception, and/or insignificant extra-solution activity:
wherein the obtained depth map stems from a 3D point cloud (general field of use, see MPEP § 2106.05(h))
When considered individually or in combination, the additional limitations and elements
of claim 9 do not amount to significantly more than the judicial exceptions for the same reasons above as to why the additional limitations do not integrate the abstract idea into a practical application.
The fact that the depth map data comes from a 3D point cloud only serves to link a judicial exception to the field of use of using 3D point cloud data. Additionally, depth map data derived from 3 dimensional data is well understood, routine, and conventional activity in the art.
Considering the claim limitations in combination and the claims as a whole does not
change this conclusion and claim 9 is ineligible under 35 U.S.C 101.
Regarding claim 10, in addition to the limitations of claim 5, claim 10 recites the limitations:
The method of claim 5, wherein the obtained depth map and/or a 3D point cloud stems from physical measurements.
The additional limitation constitutes mere general field of use, see MPEP § 2106.05(h). Merely defining the data from which a second source derives, which is only to be applied to the judicial exception, constitutes generally linking to a field of use.
When considered individually or in combination, the additional limitations and elements
of claim 9 do not amount to significantly more than the judicial exceptions for the same reasons above as to why the additional limitations do not integrate the abstract idea into a practical application. Considering the claim limitations in combination and the claims as a whole does not
change this conclusion and claim 10 is ineligible under 35 U.S.C 101.
Regarding claim 11, in addition to the limitations of claim 5, claim 11 recites the limitations:
The method of claim 5, further comprising: filtering the obtained wireframe by discarding partitions and/or junctions not satisfying a neural network prediction confidence score criterion and/or satisfying a smallness criterion.
Claim 12 contains sufficiently similar limitations to claim 5, with the additional limitations: “A device comprising: a non-transitory computer-readable data storage medium having recorded thereon: a first computer program having instructions for segmenting a building scene that when executed by a processor causes the processor to be configured to” and “and/or a second computer program having instructions for applying a neural network learnable according to the segmenting of the building scene that when executed by the processor causes the processor to be configured to” These limitations are directed to generic computing components and mere instructions to apply an exception. When considered individually or in combination, the additional limitations and elements of claim 12 do not amount to significantly more than the judicial exceptions for the same reasons above as to why the additional limitations do not integrate the abstract idea into a practical application. Considering the claim limitations in combination and the claims as a whole does not change this conclusion and claim 12 is ineligible under 35 U.S.C 101.
Regarding claim 13, in addition to the abstract idea of claim 12, claim 13 recites an additional limitation:
The device of claim 12, wherein each top-down depth map of the training dataset includes: random points, and line segments each between a respective pair of points, the line segments having random heights. (general field of use, see MPEP § 2106.05(h))
Providing information as to the structure of the training data, which is to only be applied to the judicial exception (abstract idea) of “learning … a neural network”, merely links the judicial exception to a particular field of transforming heightmap data.
When considered individually or in combination, the additional limitations and elements of claim 13 do not amount to significantly more than the judicial exceptions for the same reasons above as to why the additional limitations do not integrate the abstract idea into a practical application.
Considering the claim limitations in combination and the claims as a whole does not
change this conclusion and claim 13 is ineligible under 35 U.S.C 101.
Regarding claim 14, in addition to the abstract idea of claim 13, claim 14 recites an additional limitation:
The device of claim 13, wherein one or more top-down depth maps of the training dataset include one or more distractors, a distractor being any other object than a line segment. (general field of use, see MPEP § 2106.05(h))
Providing information as to the structure of the training data, which is to only be applied to the judicial exception (abstract idea) of “learning … a neural network”, merely links the judicial exception to a particular field of transforming heightmap data.
When considered individually or in combination, the additional limitations and elements of claim 14 do not amount to significantly more than the judicial exceptions for the same reasons above as to why the additional limitations do not integrate the abstract idea into a practical application.
Considering the claim limitations in combination and the claims as a whole does not
change this conclusion and claim 14 is ineligible under 35 U.S.C 101.
Regarding claim 15, in addition to the abstract idea of claim 14, claim 15 recites an additional limitation:
The device of claim 14, wherein one or more of the top-down depth maps include noise. (general field of use, see MPEP § 2106.05(h))
Providing information as to the structure of the training data, which is to only be applied to the judicial exception (abstract idea) of “learning … a neural network”, merely links the judicial exception to a particular field of transforming heightmap data.
When considered individually or in combination, the additional limitations and elements of claim 15 do not amount to significantly more than the judicial exceptions for the same reasons above as to why the additional limitations do not integrate the abstract idea into a practical application.
Considering the claim limitations in combination and the claims as a whole does not
change this conclusion and claim 15 is ineligible under 35 U.S.C 101.
Regarding claims 16, in addition to the limitations of claim 12, claim 16 recites the limitations:
The device of claim 12, wherein the second computer program having instructions for applying the neural network causes the processor to be further configured to compute 2D regions of the obtained depth map by extracting, from the obtained wireframe, 2D regions of which contours are formed by line segments of the obtained wireframe
The limitation “extracting, from the obtained wireframe, 2D regions of which contours are formed by line segments of the obtained wireframe” constitute an additional abstract idea (mental process). A person can identify 2d regions based on a wireframe. This would constitute mental observation and judgement.
The claim contains the additional limitation “wherein the second computer program having instructions for applying the neural network causes the processor to be further configured to compute 2D regions of the obtained depth map” which constitutes mere instructions to apply an exception with generic computing components, see MPEP § 2106.05(f).
These limitations have been considered in combination with the limitations required by the claim(s) from which this claim depends. The additional limitations are considered to constitute instructions to apply an exception (see MPEP § 2106.05(f)) and additional mental processes under step 2A prong I of the abstract idea analysis (see MPEP § 2106.04(a)(2)(III)). The additional limitations and/or additional elements do not integrate the claim limitations into a practical application (step 2A prong II), or recite significantly more than the abstract idea (step 2B). Therefore, claim 16 is ineligible under 35 U.S.C 101.
Claim 17 recites identical additional limitations to claim 16 and is ineligible under 35 U.S.C 101 for the same reasons.
Regarding claim 18, in addition to the limitations of claim 17, claim 18 recites the limitations:
The device of claim 17, wherein the processor is configured to compute the 2D regions by being further configured to:
obtain a graph
including: graph nodes each representing a junction, and graph edges each representing a line segment between two junctions represented two graph nodes, each graph edge comprising two half-edges having opposite orientations;
and determine, using the half-edges: regions of the graph delimited by graph edges and not crossed by any graph edge, and overall edge contours of the graph.
The limitation “and determine, using the half-edges: regions of the graph delimited by graph edges and not crossed by any graph edge, and overall edge contours of the graph” constitutes additional mental process. A person can make observations and judgements about a graph and perform graph operations as a mental process.
Claim 18 recites the following additional claim limitations outside the abstract idea which
only present general fields of use, mere instructions to apply an exception, and/or insignificant extra-solution activity:
wherein the processor is configured to compute the 2D regions by being further configured to: (mere instructions to apply an exception with generic computing components, see MPEP § 2106.05(f))
obtain a graph (insignificant extra-solution activity, mere data gathering, see MPEP § 2106.05(g))
having: graph nodes each representing a junction, and graph edges each representing a line segment between two junctions represented two graph nodes, each graph edge comprising two half-edges having opposite orientations; (general field of use, see MPEP § 2106.05(h))
When considered individually or in combination, the additional limitations and elements
of claim 18 do not amount to significantly more than the judicial exceptions for the same reasons above as to why the additional limitations do not integrate the abstract idea into a practical application.
Providing information as to the structure of a graph, which is to only be applied to the judicial exceptions, merely links the judicial exceptions to a particular field of use.
Considering the claim limitations in combination and the claims as a whole does not
change this conclusion and claim 18 is ineligible under 35 U.S.C 101.
Regarding claim 19, in addition to the limitations of claim 18, claim 19 recites the limitations:
The device of claim 18, wherein the processor is configured to compute the 2D regions by being further configured to:
compute, using a Shoelace formula, areas of each determined region of the graph and overall edge contour of the graph;
and discard, using the computed areas: regions having a negative area, regions having an area lower than a predefined threshold, and regions having a width lower than a predefined threshold.
The limitations “compute, using a Shoelace formula, areas of each determined region of the graph and overall edge contour of the graph” and “discard, using the computed areas: regions having a negative area, regions having an area lower than a predefined threshold, and regions having a width lower than a predefined threshold” constitute abstract ideas, specifically mental processes and mathematical equations. A person can perform the mental process of performing mathematical equations, such as the “Shoelace formula”. Additionally, a person can make the mental judgement to discard specific areas based on width.
Claim 19 contains the additional limitation “wherein the processor is configured to compute the 2D regions by being further configured to:” This limitation constitutes mere instructions to apply an exception with generic computing components, see MPEP § 2106.05(f).
When considered individually or in combination, the additional limitations and elements
of claim 19 do not amount to significantly more than the judicial exceptions for the same reasons above as to why the additional limitations do not integrate the abstract idea into a practical application.
Considering the claim limitations in combination and the claims as a whole does not change this conclusion and claim 19 is ineligible under 35 U.S.C 101.
Regarding claim 20, in addition to the limitations of claim 12, claim 20 recites:
The device of claim 12, further comprising the processor coupled to the non-transitory computer-readable data storage medium. The additional limitation constitutes mere instructions to apply an exception with generic computing components, see MPEP § 2106.05(f).
When considered individually or in combination, the additional limitations and elements
of claim 20 do not amount to significantly more than the judicial exceptions for the same reasons above as to why the additional limitations do not integrate the abstract idea into a practical application.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-6, 10-17, 20 are rejected under 35 U.S.C. 103 as being unpatentable over Zhao et al. (IDS NPL, 2021) in view of Huang et al. (IDS NPL, 2018).
Regarding claim 1, Zhao teaches the following limitations:
A computer-implemented method for segmenting a building scene, (Zhao et al. Abstract, “This paper introduces the integrally-attracted wireframe parsing (IAWP) framework to reconstruct building rooflines”)
the method comprising: obtaining a training dataset of top-down depth maps, (Zhao et al. Fig. 4) The top down depth maps are structurally similar to the images of Zhao, and Zhao teaches that further research will incorporate additional datasets such as a “digital surface model,” which is being interpreted as encoding depth data (Zhao Section 4.3, “Future research will aim to further improve the approach by training on other datasets and adding multiple data sources (e.g., a digital surface model)”)
the neural network being configured to take as input a top-down depth map of a building scene including building partitions and to output a scene wireframe including the partitions and junctions between the partitions. (Zhao et al. Section 1, “we integrate geometric line priors into deep networks for enhanced geometric feature extraction”)
The limitations of claim 1 not taught by Zhao are taught by Huang:
each depth map comprising labeled line segments (Huang et al. Section 2, “For each image, we manually labelled all the line segments associated with the scene structures.”)
and junctions between line segments; (Huang et al. Section 2, “With the labelled line segments ground truth junction locations and their branches can be easily obtained from the intersection or incidence relationships among two or more line segments in an image.”)
and learning, based on the training dataset, a neural network, (Huang et al. Section 3.2, Next we design and train a convolutional neural network (Fig. 3 bottom) to infer line information…)
Zhao et al. and Huang et al. are analogous art because they are in the same field of endeavor: Image processing to generate wireframe representations. The motivation to combine would be that Huang claims to have “convincingly shown that effectively and efficiently parsing wireframes for images of man-made environments is a feasible goal within reach.” (Huang Abstract) Huang goes on to say: “Such wireframes could benefit many important visual tasks such as feature correspondence, 3D reconstruction, vision-based mapping, localization, and navigation.” (Huang Abstract) Zhao is directed to parsing wireframes of man-made objects, and additionally, contains motivation towards using wireframes for 3D reconstruction (Zhao Abstract, “Roof shape information is essential for creating 3D building models. However, the automated extracting of roof structures from Earth observation data is a difficult task involving significant uncertainties caused by scene complexity and limited multi-source data coverage. This paper introduces the integrally-attracted wireframe parsing (IAWP) framework to reconstruct building rooflines as a planar graph from remotely sensed images with a single forward pass.”). Additionally, Huang acknowledges Hough transform approaches (Huang Related Work, “While Hough transform has the ability to accumulate information over the entire image to determine the presence of a line structure…”) and Zhao teaches a Hough transform approach (Zhao 3.2, “We integrate a Hough transform and inverse Hough transform (HT-IHT block) to combine locally learned image features with global line priors.”). Therefore, it would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to modify the wireframe generation system of Zhao with the training data of Huang.
Regarding claim 2, in addition to the limitations of claim 1, Zhao teaches the following limitations:
The computer-implemented method of claim 1, wherein each top-down depth map of the training dataset (Zhao et al. Fig. 4) The top down depth maps are structurally similar to the images of Zhao, and Zhao teaches that further research will incorporate additional datasets such as a “digital surface model,” which is being interpreted as encoding depth data (Zhao Section 4.3, “Future research will aim to further improve the approach by training on other datasets and adding multiple data sources (e.g., a digital surface model)”)
includes: random points, (Zhao 4.1, “The interior and exterior building edges are annotated as 2D planar graphs.”) The broadest reasonable interpretation of “random points” includes those derived from real training data images, and thus the points of Zhao, inherent to the “2D planar graphs” are being interpreted as such.
and line segments each between a respective pair of points, (Zhao 4.1, “The interior and exterior building edges are annotated as 2D planar graphs.”) Line segments between each pair of points is inherent to “2D planar graphs”. Further evidence of this are the planar graphs generated by Zhao (Zhao Fig. 4).
the line segments having random heights. (Zhao Fig. 4) The broadest reasonable interpretation of random height includes both real-world height, as well as pixel height, both of which read on the line segments of Zhao.
Zhao et al. and Huang et al. are analogous art because they are in the same field of endeavor: Image processing to generate wireframe representations. The motivation to combine would be that Huang claims to have “convincingly shown that effectively and efficiently parsing wireframes for images of man-made environments is a feasible goal within reach.” (Huang Abstract) Huang goes on to say: “Such wireframes could benefit many important visual tasks such as feature correspondence, 3D reconstruction, vision-based mapping, localization, and navigation.” (Huang Abstract) Zhao is directed to parsing wireframes of man-made environments, and additionally, contains motivation towards using wireframes for 3D reconstruction (Zhao Abstract, “Roof shape information is essential for creating 3D building models. However the automated extracting of roof structures from Earth observation data is a difficult task involving significant uncertainties caused by scene complexity and limited multi-source data coverage. This paper introduces the integrally-attracted wireframe parsing (IAWP) framework to reconstruct building rooflines as a planar graph from remotely sensed images with a single forward pass.”). Additionally, Huang acknowledges Hough transform approaches (Huang Related Work, “While Hough transform has the ability to accumulate information over the entire image to determine the presence of a line structure…”) and Zhao teaches a Hough transform approach (Zhao 3.2, “We integrate a Hough transform and inverse Hough transform (HT-IHT block) to combine locally learned image features with global line priors.”). Therefore, it would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to modify the wireframe generation system of Zhao with the training data of Huang.
Claim 13 recites sufficiently similar limitations to claim 2 and is rejected under 35 U.S.C. 103 for the same reasons.
Regarding claim 3, in addition to the limitations of claim 1, Zhao teaches:
The computer-implemented method of claim 1, wherein one or more top-down depth maps of the training dataset (Zhao et al. Fig. 4) The top down depth maps are structurally similar to the images of Zhao, and Zhao teaches that further research will incorporate additional datasets such as a “digital surface model,” which is being interpreted as encoding depth data (Zhao Section 4.3, “Future research will aim to further improve the approach by training on other datasets and adding multiple data sources (e.g., a digital surface model)”)
include one or more distractors, a distractor being any other object than a line segment. (Zhao et al. Fig. 4) Images of Zhao contain trees, smaller building details, shadows, roads, etc.
Zhao et al. and Huang et al. are analogous art because they are in the same field of endeavor: Image processing to generate wireframe representations. The motivation to combine would be that Huang claims to have “convincingly shown that effectively and efficiently parsing wireframes for images of man-made environments is a feasible goal within reach.” (Huang Abstract) Huang goes on to say: “Such wireframes could benefit many important visual tasks such as feature correspondence, 3D reconstruction, vision-based mapping, localization, and navigation.” (Huang Abstract) Zhao is directed to parsing wireframes of man-made environments, and additionally, contains motivation towards using wireframes for 3D reconstruction (Zhao Abstract, “Roof shape information is essential for creating 3D building models. However, the automated extracting of roof structures from Earth observation data is a difficult task involving significant uncertainties caused by scene complexity and limited multi-source data coverage. This paper introduces the integrally-attracted wireframe parsing (IAWP) framework to reconstruct building rooflines as a planar graph from remotely sensed images with a single forward pass.”). Additionally, Huang acknowledges Hough transform approaches (Huang Related Work, “While Hough transform has the ability to accumulate information over the entire image to determine the presence of a line structure…”) and Zhao teaches a Hough transform approach (Zhao 3.2, “We integrate a Hough transform and inverse Hough transform (HT-IHT block) to combine locally learned image features with global line priors.”). Therefore, it would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to modify the wireframe generation system of Zhao with the training data of Huang.
Claim 14 recites sufficiently similar limitations to claim 2 and is rejected under 35 U.S.C. 103 for the same reasons.
Regarding claim 4, in addition to the limitations of claim 2, Zhao teaches:
The computer-implemented method of claim 2, wherein one or more of the top-down depth maps include noise. (Zhao et al. Fig. 4) The broadest reasonable definition of noise includes noise in real world images.
Zhao et al. and Huang et al. are analogous art because they are in the same field of endeavor: Image processing to generate wireframe representations. The motivation to combine would be that Huang claims to have “convincingly shown that effectively and efficiently parsing wireframes for images of man-made environments is a feasible goal within reach.” (Huang Abstract) Huang goes on to say: “Such wireframes could benefit many important visual tasks such as feature correspondence, 3D reconstruction, vision-based mapping, localization, and navigation.” (Huang Abstract) Zhao is directed to parsing wireframes of man-made environments, and additionally, contains motivation towards using wireframes for 3D reconstruction (Zhao Abstract, “Roof shape information is essential for creating 3D building models. However the automated extracting of roof structures from Earth observation data is a difficult task involving significant uncertainties caused by scene complexity and limited multi-source data coverage. This paper introduces the integrally-attracted wireframe parsing (IAWP) framework to reconstruct building rooflines as a planar graph from remotely sensed images with a single forward pass.”). Additionally, Huang acknowledges Hough transform approaches (Huang Related Work, “While Hough transform has the ability to accumulate information over the entire image to determine the presence of a line structure…”) and Zhao teaches a Hough transform approach (Zhao 3.2, “We integrate a Hough transform and inverse Hough transform (HT-IHT block) to combine locally learned image features with global line priors.”). Therefore, it would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to modify the wireframe generation system of Zhao with the training data of Huang.
Claim 15 recites sufficiently similar limitations to claim 2 and is rejected under 35 U.S.C. 103 for the same reasons.
Claim 5 recites limitations sufficiently similar to claim 1, taught by Zhao in view of Huang, and are given the same analysis. Additional limitations not recited by claim 1 are taught by Zhao:
A method of applying a neural network learnable according to a computer-implemented method for segmenting a building scene, (Zhao et al. Section 1, “we integrate geometric line priors into deep networks for enhanced geometric feature extraction”)
the method of applying comprising: obtaining a top-down depth map of a building scene (Zhao et al. Fig. 4) The top down depth maps are structurally similar to the images of Zhao, and Zhao teaches that further research will incorporate additional datasets such as a “digital surface model,” which is being interpreted as encoding depth data (Zhao Section 4.3, “Future research will aim to further improve the approach by training on other datasets and adding multiple data sources (e.g., a digital surface model)”)
comprising building partitions; (Zhao et al. Fig. 4)
and applying the neural network to the obtained top-down depth map (Zhao et al. Section 1, “we integrate geometric line priors into deep networks for enhanced geometric feature extraction”)
to obtain a wireframe of the building scene, (Zhao Section 3.1, “The IAWP is an end-to-end trainable and fast parsimonious parsing method that can detect a vectorize wireframe in an input image.”)
the wireframe including the partitions and junctions between the partitions. (Zhao Section 3.1, “the IAWP consists of three components: (i) line segment and junction proposal generation…”) “partition” is being interpreted as the line segments of the wireframes.
Zhao et al. and Huang et al. are analogous art because they are in the same field of endeavor: Image processing to generate wireframe representations. The motivation to combine would be that Huang claims to have “convincingly shown that effectively and efficiently parsing wireframes for images of man-made environments is a feasible goal within reach.” (Huang Abstract) Huang goes on to say: “Such wireframes could benefit many important visual tasks such as feature correspondence, 3D reconstruction, vision-based mapping, localization, and navigation.” (Huang Abstract) Zhao is directed to parsing wireframes of man-made environments, and additionally, contains motivation towards using wireframes for 3D reconstruction (Zhao Abstract, “Roof shape information is essential for creating 3D building models. However the automated extracting of roof structures from Earth observation data is a difficult task involving significant uncertainties caused by scene complexity and limited multi-source data coverage. This paper introduces the integrally-attracted wireframe parsing (IAWP) framework to reconstruct building rooflines as a planar graph from remotely sensed images with a single forward pass.”). Additionally, Huang acknowledges Hough transform approaches (Huang Related Work, “While Hough transform has the ability to accumulate information over the entire image to determine the presence of a line structure…”) and Zhao teaches a Hough transform approach (Zhao 3.2, “We integrate a Hough transform and inverse Hough transform (HT-IHT block) to combine locally learned image features with global line priors.”). Therefore, it would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to modify the wireframe generation system of Zhao with the training data of Huang.
Regarding claim 6, in addition to the limitations of claim 5, Zhao teaches:
The method of claim 5, further comprising computing 2D regions of the obtained depth map by extracting, from the obtained wireframe, 2D regions of which contours are formed by line segments of the obtained wireframe. (Zhao et al. Fig. 4”) The 2D planar graphs of Zhao as pictured in Fig. 4 contain computed 2D regions.
Zhao et al. and Huang et al. are analogous art because they are in the same field of endeavor: Image processing to generate wireframe representations. The motivation to combine would be that Huang claims to have “convincingly shown that effectively and efficiently parsing wireframes for images of man-made environments is a feasible goal within reach.” (Huang Abstract) Huang goes on to say: “Such wireframes could benefit many important visual tasks such as feature correspondence, 3D reconstruction, vision-based mapping, localization, and navigation.” (Huang Abstract) Zhao is directed to parsing wireframes of man-made environments, and additionally, contains motivation towards using wireframes for 3D reconstruction (Zhao Abstract, “Roof shape information is essential for creating 3D building models. However the automated extracting of roof structures from Earth observation data is a difficult task involving significant uncertainties caused by scene complexity and limited multi-source data coverage. This paper introduces the integrally-attracted wireframe parsing (IAWP) framework to reconstruct building rooflines as a planar graph from remotely sensed images with a single forward pass.”). Additionally, Huang acknowledges Hough transform approaches (Huang Related Work, “While Hough transform has the ability to accumulate information over the entire image to determine the presence of a line structure…”) and Zhao teaches a Hough transform approach (Zhao 3.2, “We integrate a Hough transform and inverse Hough transform (HT-IHT block) to combine locally learned image features with global line priors.”). Therefore, it would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to modify the wireframe generation system of Zhao with the training data of Huang.
Claims 16 and 17 recite sufficiently similar limitations to claim 2 and are rejected under 35 U.S.C. 103 for the same reasons.
Regarding Claim 10, in addition to the limitations of claim 5, Zhao teaches:
The method of claim 5, wherein the obtained depth map and/or a 3D point cloud stems from physical measurements. (Zhao Fig. 4) Broadest reasonable interpretation of “physical measurements” includes the real images of Zhao. Additionally Zhao teaches that further research will incorporate additional datasets such as a “digital surface model,” which is being interpreted as encoding depth data (Zhao Section 4.3, “Future research will aim to further improve the approach by training on other datasets and adding multiple data sources (e.g., a digital surface model)”)
Zhao et al. and Huang et al. are analogous art because they are in the same field of endeavor: Image processing to generate wireframe representations. The motivation to combine would be that Huang claims to have “convincingly shown that effectively and efficiently parsing wireframes for images of man-made environments is a feasible goal within reach.” (Huang Abstract) Huang goes on to say: “Such wireframes could benefit many important visual tasks such as feature correspondence, 3D reconstruction, vision-based mapping, localization, and navigation.” (Huang Abstract) Zhao is directed to parsing wireframes of man-made environments, and additionally, contains motivation towards using wireframes for 3D reconstruction (Zhao Abstract, “Roof shape information is essential for creating 3D building models. However, the automated extracting of roof structures from Earth observation data is a difficult task involving significant uncertainties caused by scene complexity and limited multi-source data coverage. This paper introduces the integrally-attracted wireframe parsing (IAWP) framework to reconstruct building rooflines as a planar graph from remotely sensed images with a single forward pass.”). Additionally, Huang acknowledges Hough transform approaches (Huang Related Work, “While Hough transform has the ability to accumulate information over the entire image to determine the presence of a line structure…”) and Zhao teaches a Hough transform approach (Zhao 3.2, “We integrate a Hough transform and inverse Hough transform (HT-IHT block) to combine locally learned image features with global line priors.”). Therefore, it would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to modify the wireframe generation system of Zhao with the training data of Huang.
Regarding Claim 11, in addition to the limitations of claim 5, Huang teaches:
The method of claim 5, further comprising: filtering the obtained wireframe by discarding partitions and/or junctions not satisfying a neural network prediction confidence score criterion and/or satisfying a smallness criterion. (Huang 3.1.1, “If the center of a junction falls into a grid cell, that cell is responsible for detecting it. Thus, each ij-th cell predicts a confidence score cij reflecting how confident the model thinks there exists a junction in that cell.”)
Zhao et al. and Huang et al. are analogous art because they are in the same field of endeavor: Image processing to generate wireframe representations. The motivation to combine would be that Huang claims to have “convincingly shown that effectively and efficiently parsing wireframes for images of man-made environments is a feasible goal within reach.” (Huang Abstract) Huang goes on to say: “Such wireframes could benefit many important visual tasks such as feature correspondence, 3D reconstruction, vision-based mapping, localization, and navigation.” (Huang Abstract) Zhao is directed to parsing wireframes of man-made environments, and additionally, contains motivation towards using wireframes for 3D reconstruction (Zhao Abstract, “Roof shape information is essential for creating 3D building models. However the automated extracting of roof structures from Earth observation data is a difficult task involving significant uncertainties caused by scene complexity and limited multi-source data coverage. This paper introduces the integrally-attracted wireframe parsing (IAWP) framework to reconstruct building rooflines as a planar graph from remotely sensed images with a single forward pass.”). Additionally, Huang acknowledges Hough transform approaches (Huang Related Work, “While Hough transform has the ability to accumulate information over the entire image to determine the presence of a line structure…”) and Zhao teaches a Hough transform approach (Zhao 3.2, “We integrate a Hough transform and inverse Hough transform (HT-IHT block) to combine locally learned image features with global line priors.”). Therefore, it would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to modify the wireframe generation system of Zhao with the training data of Huang.
Claim 12 contains limitations sufficiently similar to those of claim 5, and these limitations will be treated in the same way. Additional limitations not covered by the analysis of claim 5 are taught by Zhao:
A device comprising: a non-transitory computer-readable data storage medium having recorded thereon: a first computer program having instructions for segmenting a building scene that when executed by a processor causes the processor to be configured to: (Zhao Section 4.2, “Our method is trained using … GeForce RTX 2080 Ti GPU device”) Additionally, application of model using a computer processor is implied by Zhao.
and/or a second computer program having instructions for applying a neural network learnable according to the segmenting of the building scene that when executed by the processor causes the processor to be configured to: (Zhao Fig. 5) Zhao implies a computer process for applying their neural network.
Zhao et al. and Huang et al. are analogous art because they are in the same field of endeavor: Image processing to generate wireframe representations. The motivation to combine would be that Huang claims to have “convincingly shown that effectively and efficiently parsing wireframes for images of man-made environments is a feasible goal within reach.” (Huang Abstract) Huang goes on to say: “Such wireframes could benefit many important visual tasks such as feature correspondence, 3D reconstruction, vision-based mapping, localization, and navigation.” (Huang Abstract) Zhao is directed to parsing wireframes of man-made environments, and additionally, contains motivation towards using wireframes for 3D reconstruction (Zhao Abstract, “Roof shape information is essential for creating 3D building models. However the automated extracting of roof structures from Earth observation data is a difficult task involving significant uncertainties caused by scene complexity and limited multi-source data coverage. This paper introduces the integrally-attracted wireframe parsing (IAWP) framework to reconstruct building rooflines as a planar graph from remotely sensed images with a single forward pass.”). Additionally, Huang acknowledges Hough transform approaches (Huang Related Work, “While Hough transform has the ability to accumulate information over the entire image to determine the presence of a line structure…”) and Zhao teaches a Hough transform approach (Zhao 3.2, “We integrate a Hough transform and inverse Hough transform (HT-IHT block) to combine locally learned image features with global line priors.”). Therefore, it would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to modify the wireframe generation system of Zhao with the training data of Huang.
Regarding Claim 20, in addition to the limitations of claim 12, Zhao teaches:
The device of claim 12, further comprising the processor coupled to the non-transitory computer-readable data storage medium. (Zhao Section 4.2, “Our method is trained using … GeForce RTX 2080 Ti GPU device”) Additionally, application of model using a computer processor is implied by Zhao.
Claim(s) 7 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Zhao et al. in view of Huang et al. as applied to claims above, and further in view of Mehta et al ("Handbook of Data Structures and Applications").
Regarding Claim 7, in addition to the limitations of claim 6 taught by Zhao in view of Haung, Mehta teaches:
The method of claim 6, wherein computing the 2D regions includes: (Mehta 65.4.2, “Two data structures are particularly well suited to working with planar subdivisions … the doubly-connected edge list (DCEL) … Both structures … allow to efficiently traverse the edges adjacent to a vertex and the edges bounding a face.”)
obtaining a graph (Mehta Fig. 18.9)
having: graph nodes each representing a junction, and graph edges each representing a line segment between two junctions represented two graph nodes, (Mehta Fig. 18.9)
each graph edge comprising two half-edges having opposite orientations; (Mehta Fig. 18.9)
and determining, using the half-edges: regions of the graph delimited by graph edges and not crossed by any graph edge, (Mehta 65.4.2, “Two data structures are particularly well suited to working with planar subdivisions … the doubly-connected edge list (DCEL) … Both structures … allow to efficiently traverse the edges adjacent to a vertex and the edges bounding a face.”) “regions of the graph delimited by graph edges and not crossed by any graph edge” is interpreted as “planar subdivisions” of Mehta.
and overall edge contours of the graph. (Mehta 65.4.2, “Two data structures are particularly well suited to working with planar subdivisions … the doubly-connected edge list (DCEL) … Both structures … allow to efficiently traverse the edges adjacent to a vertex and the edges bounding a face.”) “edge contours” are interpreted as the “edges bounding a face” of Mehta.
Zhao et al., Huang et al., and Mehta are directed to computing representations of wireframes (planar graphs). The motivation to combine would be that Huang claims to have “convincingly shown that effectively and efficiently parsing wireframes for images of man-made environments is a feasible goal within reach.” (Huang Abstract) Huang goes on to say: “Such wireframes could benefit many important visual tasks such as feature correspondence, 3D reconstruction, vision-based mapping, localization, and navigation.” (Huang Abstract) Zhao is directed to parsing wireframes of man-made environments, and additionally, contains motivation towards using wireframes for 3D reconstruction (Zhao Abstract, “Roof shape information is essential for creating 3D building models. However, the automated extracting of roof structures from Earth observation data is a difficult task involving significant uncertainties caused by scene complexity and limited multi-source data coverage. This paper introduces the integrally-attracted wireframe parsing (IAWP) framework to reconstruct building rooflines as a planar graph from remotely sensed images with a single forward pass.”). Additionally, Huang acknowledges Hough transform approaches (Huang Related Work, “While Hough transform has the ability to accumulate information over the entire image to determine the presence of a line structure…”) and Zhao teaches a Hough transform approach (Zhao 3.2, “We integrate a Hough transform and inverse Hough transform (HT-IHT block) to combine locally learned image features with global line priors.”). Additionally, Mehta teaches the doubly-connected edge list (DCEL) data structure as well suited for computing 2D regions of planar graphs. (Mehta 65.4.2, “Two data structures are particularly well suited to working with planar subdivisions … the doubly-connected edge list (DCEL) … Both structures … allow to efficiently traverse the edges adjacent to a vertex and the edges bounding a face.”) Zhao and Huang both utilize 2D planar graphs as their wireframe representations. (Zhao 4.1, “The interior and exterior building edges are annotated as 2D planar graphs”) (Huang Fig. 2) The wireframes of Huang are interpreted as encoding a 2D planar graph (Huang Section 2, “In summary, our annotation in each image includes a set of junction points … and a set of line segments…”). Therefore, it would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to modify the wireframe generation system of Zhao with the training data of Huang and furthermore with the doubly-connected edge list (and associated algorithms) of Mehta.
Claim 18 contains sufficiently similar limitations to claim 7 and is rejected under 35 U.S.C. 103 for the same reasons.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Zhao et al. in view of Huang et al. as applied to claims above, and further in view of Kao et al. (US Pub. 2021/0150726 A1).
Regarding claim 9, in addition to the limitations of claim 6 taught by Zhao in view of Huang, Kao teaches:
The method of claim 6, wherein the obtained depth map stems from a 3D point cloud, (Kao Claim 1, “An image processing method comprising: acquiring a target image comprising a depth image of a scene; determining three-dimensional (3D) point cloud data corresponding to the depth image, based on the depth image; and extracting an object included in the scene to acquire an object extraction result based on the 3D point cloud data.”) Broadest reasonable interpretation of the “3D point cloud” of claim 9 includes the point cloud represented by the “depth map” of claim 9 and would be equivalent to the point cloud and depth map of Kao.
and the method further comprises: projecting the computed 2D regions on the 3D point cloud, thereby obtaining a 3D segmentation of the building scene. (Kao Claim 11, “The image processing method of claim 1, further comprising: determining a 3D detection result of an object included in the target image, based on the object extraction result, wherein the 3D detection result comprises at least one of a 3D pose result and a 3D segmentation result.”)
Zhao et al., Huang et al., and Kao et al. are analogous art because they are in the same field of endeavor: Image processing to generate wireframe representations. The motivation to combine would be that Huang claims to have “convincingly shown that effectively and efficiently parsing wireframes for images of man-made environments is a feasible goal within reach.” (Huang Abstract) Huang goes on to say: “Such wireframes could benefit many important visual tasks such as feature correspondence, 3D reconstruction, vision-based mapping, localization, and navigation.” (Huang Abstract) Zhao is directed to parsing wireframes of man-made environments, and additionally, contains motivation towards using wireframes for 3D reconstruction (Zhao Abstract, “Roof shape information is essential for creating 3D building models. However, the automated extracting of roof structures from Earth observation data is a difficult task involving significant uncertainties caused by scene complexity and limited multi-source data coverage. This paper introduces the integrally-attracted wireframe parsing (IAWP) framework to reconstruct building rooflines as a planar graph from remotely sensed images with a single forward pass.”). Kao is also directed toward 2D image segmentation for 3D object reconstruction (Kao Fig. 18). Additionally, like Zhao and Huang, Kao uses a Neural Network model to generate the image segmentations. Therefore, it would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to modify the wireframe generation system of Zhao with the training data of Huang and furthermore with the 3D point cloud of Kao.
Allowable Subject Matter
Claims 8 and 19 are objected to as being dependent upon a rejected base claim but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims, and amended to overcome the above 101 and 112 rejections.
The limitations directed to discarding geometry based on an area calculation, within the specific context of depth map region identification, are not found in the prior art of record.
Zhao et al. teaches training and applying a neural network to identify partitions and junctions in a building scene, and generating a corresponding wireframe (Zhao et al. Section 1, “we integrate geometric line priors into deep networks for enhanced geometric feature extraction”). Zhao et al. does not teach discarding regions based on calculating areas.
Huang et al. teaches training a neural network with labeled wireframe data, and applying the neural network to identify partitions and junctions in an image, and generating a corresponding wireframe (Huang et al. Section 3, “Utilizing the dataset we have, we here design new, end-to-end trainable CNNs for detecting junctions and lines, respectively, and then merge them into a complete wireframe.”). Huang et al/ does not teach discarding regions based on calculating areas.
Mehta teaches using a doubly-connected edge list for calculating regions of a 2D planar graph (Mehta 65.4.2, “Two data structures are particularly well suited to working with planar subdivisions … the doubly-connected edge list (DCEL) … Both structures … allow to efficiently traverse the edges adjacent to a vertex and the edges bounding a face.”). Mehta does not teach training and applying a neural network to identify partitions and junctions in a building scene, generating a corresponding wireframe, and discarding regions based on calculating areas.
Kao et al. teaches applying a neural network to take as input depth data and generating wireframes, partitioning wireframes into 2D regions, generating a point cloud, and generating corresponding 3D regions (Kao et al. Fig. 7). Kao et al. does not teach discarding regions based on calculating areas.
Thus, the subject matter of claims 8 and 19 are indicated as allowable over prior art of record.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Rejeb Sfar et al. (US 2019/0205485 A1) teaches generating 3d models from building floorplans.
Mehr et al. (US 2018/0330184 A1) teaches generating a building model from a depth sensor and point cloud.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HENRY GOLD whose telephone number is (571)270-7171. The examiner can normally be reached Monday-Friday (9:00 am - 5:00pm) ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Rehana Perveen can be reached at 571-272-3676. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HENRY JOYNER GOLD/Examiner, Art Unit 2189
/REHANA PERVEEN/Supervisory Patent Examiner, Art Unit 2189
Prosecution Insights