DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
The following title is suggested: Architecture and Operation .
Claim Objections
Claim objected to because of the following informalities:
Claims 1, 13, and 20 use the language, “one or more … models”. However, in the last limitation of claims 1 and 13, and the sixth limitation in claim 20, the phrase “…the displacement do not match the … models” is used. This appears that it should be “the displacement do not match the one or more … models”.
Claims 2, 13, and 14 use the language, “… one more …”. This appears that it should be “… one or more…” .
Claim 10 uses the language, “the first pose represented in a second local coordinate system different from the first local coordinate system”. Based on the Examiner’s understanding of the invention, the first and second poses should each have their own local coordinate system. Thus, this appears that the language should be, “the second pose represented in a second local coordinate system different from the first local coordinate system”
Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 20 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter because a learning processor, according to specification of the instant application, can be software. This would result in independent claim 20 being directed towards software per se, rather than a machine.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-5, and 13-17 is/are rejected under 35 U.S.C. 102(a)(1) and (a)(2) as being anticipated US 20170220887 A1 by Fathi et al., hereafter Fathi.
Regarding independent claim 1, Fathi teaches:
A computer-implemented method comprising:
receiving ( “the software elicits and receives selection of specific object(s) that are recognized in the scene or for which other information can be provided (e.g., measurements, dimensions, topology, labels, semantics).”) one or more features of an object (e.g., measurements, dimensions, topology, labels, semantics); (Fathi Paragraph [0144])
receiving (“3D information is provided at 125”), by a first learning processor ((Fathi Paragraph [0165]) software embodied in a medium for use by a processor 312), a current pose (point in point clouds or wireframes, included in provided topology information) associated with the one or more features of the object (“includes information about all or part of the selected object(s)”); (Fathi Paragraph [0146])
converting the current pose to a displacement (relative location information) relative to a previous pose received prior to the current pose (“where such relative location information is provided by topology information about two or more objects.”); (Fathi Paragraph [0117])
responsive ((Fathi Paragraph [0098]) “The output of the cross referencing procedure between the 2D and 3D information can also be characterized as validated 2D and 3D information”) to determining (validated/verified) that the one or more features and the displacement match one or more models ((Fathi Paragraph [0157]) “the measurements, dimensions, geometries, and topography information can be compared with a reference object and/or verified information derived from an object library (such as by providing known size information for the object as matched with pertinent information in the object library) so as to provide validated/verified labeled information 299.”), updating evidence values (defining bounding boxes for projective views) of the one or more models; ((Fathi Paragraph [0103]) “each view serves as evidence towards the determination of the object predictive label”)and
responsive to determining that the one or more features and the displacement do not match the models, ((Fathi Paragraph [0158]) “If the semantic information is not in agreement with the prediction information,”) generating ((Fathi Paragraph [0100]) “the present methodology can generate various representations for the objects of interest to be further processed for the determination of their class” the prediction in Fathi Paragraph [0158] could be for a new representation of the object) a new model (representations for the objects of interest) corresponding to (taking in account) the displacement (geometries and topography information) and associated with the one or more features (measurement, and dimensions information). ((Fathi Paragraph [0157] Equating semantic information to measurements, dimensions, geometries, and topography information due to the language of “In this regard” in response to processing of semantic information))
Regarding claim 2, Fathi teaches the method in claim 1 and additionally teaches:
wherein each of the models indicates one more of a candidate identity (label) of the object ((Fathi Paragraph [0174])“A label can be generated for the selected object”), a candidate pose ((Fathi Paragraph [0099]) estimated geometric and topological information) of the object or a candidate state ((Fathi Paragraph [0099]) estimated dimensions) of the object.
Regarding claim 3, Fathi teaches the method in claim 1 and additionally teaches:
storing ((Fathi Paragraph [0115]) stored in object libraries) the new model ((Fathi Paragraph [0100]) representations for the objects of interest) in the first learning processor ((Fathi Paragraph [0121]) stored on machines) responsive to generating the new model ((Fathi Paragraph [0115]) “derived from this object recognition”). ((Fathi Paragraph [0100]) “further object recognition improvements are achievable with the methodology herein. In this regard, the present methodology can generate various representations for the objects of interest to be further processed for the determination of their class.”);(Fathi Paragraph [0115]) “The object information derived from this object recognition can further be stored in object libraries relevant to one or more locations or environments for later use.”) ((Fathi Paragraph [0121]) “For example, the libraries of object information may all be stored within a single relational database management system, e.g., as different tables of a single database, or may be stored on separate machines”)
Regarding claim 4, Fathi teaches the method in claim 1 and additionally teaches:
generating of the new model ((Fathi Paragraph [0100]) object recognition improvement to generate various representations for the objects) is performed in an unsupervised manner (“the object recognition and/or labeling process can become substantially unsupervised”). (Fathi Paragraph [0113])
Regarding claim 5, Fathi teaches the method in claim 1 and additionally teaches:
each of the models (representations of one or more surfaces) is a graph model (wireframe) including a plurality of nodes (vertices) corresponding to different parts of a candidate object (structure), each of the nodes associated with one or more features ((Fathi Paragraph [0073]) Wireframes incorporate information about the scene. Scene information comprises measurements, depth and the like). (Fathi Paragraph [0071])
Regarding independent claim 13, Fathi teaches:
A non-transitory computer readable storage medium storing instructions thereon, the instructions when executed by one or more processors cause the one more processors to: ((Fathi Paragraph [0165]) “Any logic or application described herein, including the information extraction application 321, that comprises software or instructions can be embodied in any non-transitory computer-readable medium for use by or in connection with an instruction execution system such as, for example, a processor 312 in a computer system or other system.”)
receive ( “the software elicits and receives selection of specific object(s) that are recognized in the scene or for which other information can be provided (e.g., measurements, dimensions, topology, labels, semantics).”) one or more features of an object (e.g., measurements, dimensions, topology, labels, semantics); (Fathi Paragraph [0144])
receive (“3D information is provided at 125”) a current pose (point in point clouds or wireframes, included in provided topology information) associated with the one or more features of the object (“includes information about all or part of the selected object(s)”); (Fathi Paragraph [0146])
convert the current pose to a displacement (relative location information) relative to a previous pose received prior to the current pose (“where such relative location information is provided by topology information about two or more objects.”); (Fathi Paragraph [0117])
responsive ((Fathi Paragraph [0098]) “The output of the cross referencing procedure between the 2D and 3D information can also be characterized as validated 2D and 3D information”) to determining (validated/verified) that the one or more features and the displacement match one or more stored((Fathi Paragraph [0115]) stored in object libraries) models, ((Fathi Paragraph [0157]) “the measurements, dimensions, geometries, and topography information can be compared with a reference object and/or verified information derived from an object library (such as by providing known size information for the object as matched with pertinent information in the object library) so as to provide validated/verified labeled information 299.”), update evidence values (defining bounding boxes for projective views) of the one or more models; ((Fathi Paragraph [0103]) “each view serves as evidence towards the determination of the object predictive label”)and
responsive to determining that the one or more features and the displacement do not match the stored((Fathi Paragraph [0115]) stored in object libraries) models, ((Fathi Paragraph [0158]) “If the semantic information is not in agreement with the prediction information,”) generate ((Fathi Paragraph [0100]) “the present methodology can generate various representations for the objects of interest to be further processed for the determination of their class” the prediction in Fathi Paragraph [0158] could be for a new representation of the object) a new model (representations for the objects of interest) corresponding to (taking in account) the displacement (geometries and topography information) and associated with the one or more features (measurement, and dimensions information). ((Fathi Paragraph [0157] Equating semantic information to measurements, dimensions, geometries, and topography information due to the language of “In this regard” in response to processing of semantic information))
Regarding claim 14, Fathi teaches the non-transitory computer readable storage medium in claim 13 and additionally teaches:
wherein each of the stored models indicates one more of a candidate identity (label) of the object ((Fathi Paragraph [0174])“A label can be generated for the selected object”), a candidate pose ((Fathi Paragraph [0099]) estimated geometric and topological information) of the object or a candidate state ((Fathi Paragraph [0099]) estimated dimensions) of the object.
Regarding claim 15, Fathi teaches the non-transitory computer readable storage medium in claim 13 and additionally teaches:
the instructions further cause the one or more processors to store ((Fathi Paragraph [0115]) stored in object libraries) the new model ((Fathi Paragraph [0100]) representations for the objects of interest) responsive to generating the new model ((Fathi Paragraph [0115]) “derived from this object recognition”). ((Fathi Paragraph [0100]) “further object recognition improvements are achievable with the methodology herein. In this regard, the present methodology can generate various representations for the objects of interest to be further processed for the determination of their class.”);(Fathi Paragraph [0115]) “The object information derived from this object recognition can further be stored in object libraries relevant to one or more locations or environments for later use.”) ((Fathi Paragraph [0121]) “For example, the libraries of object information may all be stored within a single relational database management system, e.g., as different tables of a single database, or may be stored on separate machines”)
Regarding claim 16, Fathi teaches the non-transitory computer readable storage medium in claim 13 and additionally teaches:
generating of the new model ((Fathi Paragraph [0100]) object recognition improvement to generate various representations for the objects) is performed in an unsupervised manner (“the object recognition and/or labeling process can become substantially unsupervised”). (Fathi Paragraph [0113])
Regarding claim 17, Fathi teaches the non-transitory computer readable storage medium in claim 13 and additionally teaches:
each of the stored models (representations of one or more surfaces) is a graph model (wireframe) including a plurality of nodes (vertices) corresponding to different parts of a candidate object (structure), each of the nodes associated with one or more features ((Fathi Paragraph [0073]) Wireframes incorporate information about the scene. Scene information comprises measurements, depth and the like). (Fathi Paragraph [0071])
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 6 and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Fathi, and further in view of US 20210012523 A1 by Zhou et al., hereafter Zhou.
Regarding claim 6, Fathi teaches the method disclosed in claim 1.
Fathi does not explicitly disclose:
obtaining converted displacements that coincide with poses of the one or more models by rotating the displacement; and
using the converted displacement to determine locations of the models for comparing with the one or more features associated with the object.
Zhou teaches:
obtaining converted (adjusted) displacements that coincide (in accordance) with poses (coordinates) of the one or more models (three-dimensional coordinate system) by rotating the displacement (adjusting the initial rotation matrix); and (Zhou Paragraph [0116])
using the converted displacement (displacement vectors and rotation matrix after keypoint detection processing such as done in Zhou Paragraph [0116]) to determine locations (estimated coordinates of each keypoint) of the models (images) for comparing with the one or more features associated with the object. ((Zhou Fig. 4.) (Zhou Paragraph [0130] keypoint detection processing uses the displacement vectors and rotation matrix))
Zhou and Fathi are analogous art because they are in the same field of endeavor: object detection and recognition.
Thus, it would have been obvious to a person having ordinary skill in the art, before the effective filing date, having the references in front of them, to have modified Fathi, which already teaches receiving pose information and displacement information, but does not explicitly disclose rotating the displacement, by adjusting the rotation of the displacements so that it coincides with poses, in order to determine coordinates for all of the important points, as taught by Zhou. The motivation for this would have been to fix instances where the matching of the initial output pose and the real pose based on an initial displacement has errors (Zhou Paragraph [0003]). This application to Fathi of the known technique of rotating the displacement to obtain converted displacements coinciding with poses of the models and using the converted displacements to determine locations of the models, as taught by Zhou, would yield the predictable result that is disclosed in claim 6 of the instant application.
Regarding claim 18, Fathi teaches the non-transitory computer readable storage medium disclosed in claim 13.
Fathi does not explicitly disclose:
obtaining converted displacements that coincide with poses of the one or more models by rotating the displacement; and
using the converted displacement to determine locations of the models for comparing with the one or more features associated with the object.
Zhou teaches:
obtain converted (adjusted) displacements that coincide (in accordance) with poses (coordinates) of the one or more models (three-dimensional coordinate system) by rotating the displacement (adjusting the initial rotation matrix); and (Zhou Paragraph [0116])
use the converted displacement (displacement vectors and rotation matrix after keypoint detection processing such as done in Zhou Paragraph [0116]) to determine locations (estimated coordinates of each keypoint) of the stored models (images) for comparing with the one or more features associated with the object. ((Zhou Fig. 4.) (Zhou Paragraph [0130] keypoint detection processing uses the displacement vectors and rotation matrix))
Zhou and Fathi are analogous art because they are in the same field of endeavor: object detection and recognition.
Thus, it would have been obvious to a person having ordinary skill in the art, before the effective filing date, having the references in front of them, to have modified Fathi, which already teaches receiving pose information and displacement information, but does not explicitly disclose rotating the displacement, by adjusting the rotation of the displacements so that it coincides with poses, in order to determine coordinates for all of the important points, as taught by Zhou. The motivation for this would have been to fix instances where the matching of the initial output pose and the real pose based on an initial displacement has errors (Zhou Paragraph [0003]). This application to Fathi of the known technique of rotating the displacement to obtain converted displacements coinciding with poses of the models and using the converted displacements to determine locations of the models, as taught by Zhou, would yield the predictable result that is disclosed in claim 18 of the instant application.
Claim(s) 7-12, and 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Fathi, and further in view of US 20210374578 A1 by Hawkins et al., hereafter Hawkins.
Regarding claim 7, Fathi teaches the method disclosed in claim 1.
Fathi does not explicitly disclose:
receiving, by the first learning processor, a lateral vote signal indicating one or more of (i) a candidate identity of the object, (ii) a candidate pose of the object, or (iii) a candidate state of the object, as predicted by a second learning processor at a same hierarchical level as the first learning processor; and
adjusting the evidence values according to the lateral vote signal.
Hawkins teaches:
receiving, by the first learning processor (output processor A), a lateral vote signal (inter-lateral signals 1682) indicating one or more of (i) a candidate identity of the object ((Hawkins Paragraph [0093]) activated cells representing presence of a particular object), (ii) a candidate pose of the object, or (iii) a candidate state of the object, as predicted by a second learning processor (output processor B) at a same hierarchical level as the first learning processor; (Hawkins Fig.16 and Paragraph [0189]) and
adjusting the evidence values (increasing or decreasing permanence values) according to the lateral vote signal (due to lateral signals). (Hawkins Paragraph [0209])
Hawkins and Fathi are analogous art because they are in the same field of endeavor: object detection and recognition.
Thus, it would have been obvious to a person having ordinary skill in the art, before the effective filing date, having the references in front of them, to have modified Fathi, which already teaches adjusting evidence values towards determination of a label but does not explicitly disclose a lateral vote signal, to include the lateral signal taught by Hawkins. The motivation for this would have been to reinforce representations of common objects (Hawkins Paragraph [0095]). This application of the known technique of sending a lateral signal to a processor on the same hierarchical level for adjusting evidence values, as taught by Hawkins, to Fathi would yield the predictable result that is disclosed in claim 7 of the instant application.
Regarding claim 8, Fathi, in view of Hawkins, teaches the method disclosed in claim 7.
Fathi does not explicitly disclose:
receiving, by the first learning processor, a downstream signal indicating (i) a candidate identity of the object, (ii) a candidate pose of the object, or (iii) a candidate state of the object, as predicted by a third learning processor at a hierarchical level higher than the first learning processor; and
adjusting the evidence values according to the downstream signal.
Hawkins teaches:
receiving, by the first learning processor (output processor L), a downstream signal (inter-layer voting signal 1664) indicating one or more of (i) a candidate identity of the object ((Hawkins Paragraph [0093]) activated cells representing presence of a particular object), (ii) a candidate pose of the object, or (iii) a candidate state of the object, as predicted by a third learning processor (output processor K) at a hierarchical level higher than the first learning processor; (Hawkins Fig.16 and Paragraph [0187]) and
adjusting the evidence values (adjusting permanence values) according to the downstream signal (through inter-layer voting connections). (Hawkins Paragraph [0191])
Hawkins and Fathi are analogous art because they are in the same field of endeavor: object detection and recognition.
Thus, it would have been obvious to a person having ordinary skill in the art, before the effective filing date, having the references in front of them, to have modified Fathi, which already teaches adjusting evidence values towards determination of a label but does not explicitly disclose a downstream signal, to include the inter-layer voting signal taught by Hawkins. The motivation for this would have been to reinforce representations of common objects (Hawkins Paragraph [0187]). This application of the known technique of sending an inter-layer voting signal to a processor from a higher hierarchical level for adjusting evidence values, as taught by Hawkins, to Fathi would yield the predictable result that is disclosed in claim 8 of the instant application.
Regarding claim 9, Fathi, in view of Hawkins, teaches the method disclosed in claim 7.
Fathi does not explicitly disclose:
receiving, by a first sensor processor, first sensory input data;
identifying, by the first sensor processor, the one or more features by processing the first sensory input data;
receiving, by the first sensor processor, a first raw pose associated with the first sensory input data, the first pose represented in a first local coordinate system;
processing the first raw pose into a first converted pose represented in a common coordinate system; and
sending the one or more features and the first converted pose from the first sensor processor to the first learning processor.
Hawkins teaches:
receiving, by a first sensor processor(an input processor 410A), first sensory input data(sensory input data 110A) ((Hawkins Fig. 4) sensory input data 110A is provided to input processors 410A from Controller 422);
identifying (activating cells), by the first sensor processor, the one or more features by processing the first sensory input data; ((Hawkins Paragraph [0104]) The input processor activates cells associated with the feature described in the sensory input data)
receiving, by the first sensor processor, a first raw pose (location information120A) associated with the first sensory input data ((Hawkins Paragraph [0069]) “The location information 120 represents a location associated with sensors generating the sensory input data 110”), the first pose represented in a first local coordinate system (cell);
processing ((Hawkins Paragraph [0118])feeding the location information to the cells) the first raw pose into a first converted pose (columns of cells that have received column activation signals) represented in a common coordinate system (column of cells); and (Hawkins Paragraph [0120] and Fig. 6)
sending ((Hawkins Paragraph [0125]) sending the feedforward signal of the input processor to the output processor) the one or more features and the first converted pose from the first sensor processor to the first learning processor ((Hawkins Fig. 4) an output processor 412A).
Hawkins and Fathi are analogous art because they are in the same field of endeavor: object detection and recognition.
Thus, it would have been obvious to a person having ordinary skill in the art, before the effective filing date, having the references in front of them, to have modified Fathi, which already teaches obtaining sensor information and sending it to be processed but does not explicitly disclose the specific structure as stated in this claim, to include receiving and processing feature and location information and sending it to be used in a learning processor, as taught by Hawkins. The motivation for this would have been to define the system so that the set of candidate objects is consistent (Hawkin [0066]). This application of the known technique of receiving and processing feature and location information and sending the results to a further processor, as taught by Hawkins, to Fathi would yield the predictable result that is disclosed in claim 9 of the instant application.
Regarding claim 10, Fathi, in view of Hawkins, teaches the method disclosed in claim 7.
Fathi does not explicitly disclose:
receiving, by a second sensor processor, second sensory input data;
identifying, by the second sensor processor, one or more additional features by processing the second sensory input data;
receiving, by the second sensor processor, a second raw pose associated with the second sensory input data, the first pose represented in a second local coordinate system different from the first local coordinate system;
processing the second raw pose into a second converted pose represented in the common coordinate system; and
sending the one or more additional features and the second converted pose from the second sensor processor to the second learning processor.
Hawkins teaches:
receiving, by a second sensor processor(an input processor 410B), second sensory input data(sensory input data 110B) ((Hawkins Fig. 4) sensory input data 110B is provided to input processors 410B from Controller 422);
identifying (activating cells), by the second sensor processor, the one or more additional features by processing the second sensory input data; ((Hawkins Paragraph [0104]) The input processor activates cells associated with the feature described in the sensory input data)
receiving, by the second sensor processor, a second raw pose (location information120A) associated with the second sensory input data ((Hawkins Paragraph [0069]) “The location information 120 represents a location associated with sensors generating the sensory input data 110”), the second pose represented in a second local coordinate system (cell) different from the first local coordinate system;
processing ((Hawkins Paragraph [0118])feeding the location information to the cells) the second raw pose into a second converted pose (columns of cells that have received column activation signals) represented in a common coordinate system (column of cells); and (Hawkins Paragraph [0120] and Fig. 6)
sending ((Hawkins Paragraph [0125]) sending the feedforward signal of the input processor to the output processor) the one or more additional features and the second converted pose from the second sensor processor to the second learning processor ((Hawkins Fig. 4) an output processor 412A).
Hawkins and Fathi are analogous art because they are in the same field of endeavor: object detection and recognition.
Thus, it would have been obvious to a person having ordinary skill in the art, before the effective filing date, having the references in front of them, to have modified Fathi, which already teaches obtaining sensor information and sending it to be processed but does not explicitly disclose the specific structure as stated in this claim, to include receiving and processing feature and location information and sending it to be used in a learning processor, as taught by Hawkins. The motivation for this would have been to define the system so that the set of candidate objects is consistent (Hawkin [0066]). This application of the known technique of receiving and processing feature and location information and sending the results to a further processor, as taught by Hawkins, to Fathi would yield the predictable result that is disclosed in claim 10 of the instant application.
Regarding claim 11, Fathi, in view of Hawkins, teaches the method disclosed in claim 7.
Fathi does not explicitly disclose:
Hawkins teaches:
receiving a sensor signal (receiving feedforward signal) by the second learning processor (output processor B), the sensor signal indicating one or more additional features of the object and an additional pose (location representation) (indicating feature-location representations); (Hawkins Paragraph [0092])
determining one or more of the candidate poses and candidate identities (determines a set of candidate objects…) of the object associated with the sensor signal (…based on the feedforward signals) by comparing with models stored in the second learning processor ((Hawkins Paragraph [0158]) matching known object representations and generating the set of candidate objects); (Hawkins Paragraph [0093]) and
generating the lateral vote signal (The output processor generates and provides the activation states of the cells as a lateral signal) based on the comparing with the models stored in the second learning processor (based on the feedforward signal). (Hawkins Paragraph [0105])
Hawkins and Fathi are analogous art because they are in the same field of endeavor: object detection and recognition.
Thus, it would have been obvious to a person having ordinary skill in the art, before the effective filing date, having the references in front of them, to have modified Fathi, which already teaches receiving sensor data and determining if it matches models stored in libraries in the processor but does not explicitly disclose generating a lateral vote signal, to include the generation of a lateral signal based on the determinations using the received information taught by Hawkins. The motivation for this would have been to generate signals that can be used to reinforce representations of common objects (Hawkins Paragraph [0187]). This application of the known technique of generating a lateral signal, as taught by Hawkins, to Fathi would yield the predictable result that is disclosed in claim 11 of the instant application.
Regarding claim 12, Fathi teaches the method disclosed in claim 1, and additionally teaches:
generating, by the first learning processor, an action output to a motor controller (feedback to control motors) to cause one or more actuators to transition to a target pose (moving components), the one or more actuators associated with a sensor for detecting the one or more features (for sensing position). (Fathi Paragraph [0170])
Hawkins better teaches:
generating, by the first learning processor, an action output to a motor controller to cause one or more actuators ((Hawkins Paragraph [0071]) The inference system produces an output representing control signals for machines) to transition to a target pose of the one or more actuators, the one or more actuators associated with a sensor for detecting the one or more features ((Hawkins Paragraph [0092]) the controller sends operation signals to actuators associated with the movement of sensors that provide sensor inputs).
Hawkins and Fathi are analogous art because they are in the same field of endeavor: object detection and recognition.
Thus, it would have been obvious to a person having ordinary skill in the art, before the effective filing date, having the references in front of them, to have modified Fathi, which teaches control systems including control motors that cause actuators to move, to use the output from an inference system to control actuators that control movement of sensors. The motivation for this would have been to directly connect the output of the learning processor with controlling a machine associated with sensors. This simple substitution of the method of motor control as taught by Hawkins into the idea of controlling motors as in Fathi, would obtain the predictable result that is disclosed in claim 12 of the instant application.
Regarding claim 19, Fathi teaches the non-transitory computer readable storage medium disclosed in claim 13.
Fathi does not explicitly disclose:
a first processor of the one or more processors to receive a signal indicating one or more of (i) a candidate identity of the object, (ii) a candidate pose of the object, or (iii) a candidate state of the object, as predicted by a second processor of the one or more processors; and
adjust the evidence values by the first processor according to the received signal.
Hawkins teaches:
a first processor of one or more processors (output processor A) to receive a signal (inter-lateral signals 1682) indicating one or more of (i) a candidate identity of the object ((Hawkins Paragraph [0093]) activated cells representing presence of a particular object), (ii) a candidate pose of the object, or (iii) a candidate state of the object, as predicted by a second processor (output processor B)) of the one or more processors; (Hawkins Fig.16 and Paragraph [0189]) and
adjusting the evidence values (increasing or decreasing permanence values) according to the received signal (due to lateral signals). (Hawkins Paragraph [0209])
Hawkins and Fathi are analogous art because they are in the same field of endeavor: object detection and recognition.
Thus, it would have been obvious to a person having ordinary skill in the art, before the effective filing date, having the references in front of them, to have modified Fathi, which already teaches adjusting evidence values towards determination of a label but does not explicitly disclose a receiving a signal from a second processor, to include the receiving a signal from a second processor, as taught by Hawkins. The motivation for this would have been to reinforce representations of common objects (Hawkins Paragraph [0095]). This application of the known technique of sending a signal from a second processor to a first processor for adjusting evidence values, as taught by Hawkins, to Fathi would yield the predictable result that is disclosed in claim 19 of the instant application.
Regarding independent claim 20, Fathi teaches:
A computing device comprising:
a first learning processor configured to: ((Fathi Paragraph [0165]) “Any logic or application described herein, including the information extraction application 321, that comprises software or instructions can be embodied in any non-transitory computer-readable medium for use by or in connection with an instruction execution system such as, for example, a processor 312 in a computer system or other system.”)
receive ( “the software elicits and receives selection of specific object(s) that are recognized in the scene or for which other information can be provided (e.g., measurements, dimensions, topology, labels, semantics).”) one or more features of an object (e.g., measurements, dimensions, topology, labels, semantics); (Fathi Paragraph [0144])
receive (“3D information is provided at 125”) a current pose (point in point clouds or wireframes, included in provided topology information) associated with the one or more features of the object (“includes information about all or part of the selected object(s)”); (Fathi Paragraph [0146])
convert the current pose to a displacement (relative location information) relative to a previous pose received prior to the current pose (“where such relative location information is provided by topology information about two or more objects.”), (Fathi Paragraph [0117])
responsive ((Fathi Paragraph [0098]) “The output of the cross referencing procedure between the 2D and 3D information can also be characterized as validated 2D and 3D information”) to determining (validated/verified) that the one or more features and the displacement match one or more first models stored in the first learning processor ((Fathi Paragraph [0115]) stored in object libraries), ((Fathi Paragraph [0157]) “the measurements, dimensions, geometries, and topography information can be compared with a reference object and/or verified information derived from an object library (such as by providing known size information for the object as matched with pertinent information in the object library) so as to provide validated/verified labeled information 299.”), update evidence values (defining bounding boxes for projective views) of the one or more models, ((Fathi Paragraph [0103]) “each view serves as evidence towards the determination of the object predictive label”)and
responsive to determining that the one or more features and the displacement do not match the stored((Fathi Paragraph [0115]) stored in object libraries) models, ((Fathi Paragraph [0158]) “If the semantic information is not in agreement with the prediction information,”) generate ((Fathi Paragraph [0100]) “the present methodology can generate various representations for the objects of interest to be further processed for the determination of their class” the prediction in Fathi Paragraph [0158] could be for a new representation of the object) a new model (representations for the objects of interest) corresponding to (taking in account) the displacement (geometries and topography information) and associated with the one or more features (measurement, and dimensions information). ((Fathi Paragraph [0157] Equating semantic information to measurements, dimensions, geometries, and topography information due to the language of “In this regard” in response to processing of semantic information));
Fathi does not explicitly disclose:
a second learning processor (output processor B) configured to:
receive one or more additional features of the object and an additional pose,
determine one or more of candidate poses or candidate identities of the object by comparing the one or more features and the additional pose with second models stored in the second learning processor,
generate a lateral vote signal based on the comparing with the models stored in the second learning processor, and
send the lateral vote signal to the first learning processor to adjust the evidence values or generate the new model.
Hawkins teaches:
a second learning processor ((Hawkins Fig.16 and Paragraph [0189]) output processor B) configured to:
receive (receiving feedforward signal) one or more additional features of the object and an additional pose(location representation) (indicating feature-location representations), (Hawkins Paragraph [0092])
determine one or more of candidate poses or candidate identities ((Hawkins Paragraph [0093]) determines a set of candidate objects…) of the object by comparing the one or more features and the additional pose with second models stored in the second learning processor((Hawkins Paragraph [0158]) matching known object representations and generating the set of candidate objects),
generate a lateral vote signal ((Hawkins Paragraph [0105]) The output processor generates and provides the activation states of the cells as a lateral signal) based on the comparing with the models stored in the second learning processor((Hawkins Paragraph [0158]) matching known object representations and generating the set of candidate objects), and
send the lateral vote signal (provides as a lateral signal) to the first learning processor to adjust the evidence values ((Hawkins Paragraph [0209]) increasing or decreasing permanence values) or generate the new model.
Hawkins and Fathi are analogous art because they are in the same field of endeavor: object detection and recognition.
Thus, it would have been obvious to a person having ordinary skill in the art, before the effective filing date, having the references in front of them, to have modified Fathi, which already teaches receiving information, processing the received information, and using the processed information for adjusting evidence values towards determination of a label but does not explicitly disclose a using or generating a lateral vote signal with a second processor, to include generating the lateral signal using a second learning processor, and sending the lateral signal back to the first for use in adjusting evidence values, as taught by Hawkins. The motivation for this would have been to reinforce representations of common objects (Hawkins Paragraph [0095]). This application of the known technique of generating a lateral signal, and sending the lateral signal to a processor on the same hierarchical level for adjusting evidence values, as taught by Hawkins, to Fathi would yield the predictable result that is disclosed in claim 20 of the instant application.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Patents and/or related publications are cited in the Notice of References Cited (Form PTO-892) attached to this action to further show the state of the art with respect to object detection, object recognition, and hierarchical systems.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DYLAN H LAI whose telephone number is (571)272-8628. The examiner can normally be reached Monday - Friday 7:30am-5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tamara Kyle can be reached at 5712524241. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/D.H.L./Examiner, Art Unit 2144
/TAMARA T KYLE/Supervisory Patent Examiner, Art Unit 2144
