DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
In the Reply filed 22 May 2026 Applicant argues:
The term "single-snap stereo fusion device" refers to stereo vision devices in which a single-snap acquires all necessary data (e.g., depth and/or color, etc.) in a single photographic exposure. Instead of taking multiple photos over time or scanning laser beams across a room, it uses a single shot to gather light from multiple angles and immediately fuse that data into a 3D model.
Applicant further argues that a person of skill in the art would understand the term “single-snap stereo fusion device”.
In response, the instant specification as filed fails to provide the details listed in the argument quoted above but is, instead, included with a laundry list of terms for the spatial sensor. Furthermore, Applicant has provided no evidence that “single-snap stereo fusion device” is a well-known term of art. To the contrary, a key word search of all the documents in the US-PGPUB, USPAT, FIT, FPRS, EPO and JPO databases reveals a single solitary hit for this term which is the published instant application.
Upon further reflection on this issue, the rejections have been expanded to include rejections under both 112(a) and 112(b).
In the Reply filed 22 May 2026 Applicant admits that Fig. 6 of Watanabe uses a 3D LiDAR sensor to aid in determining the position of an object detected in a 2D image from a camera but questions whether the pre-configuration of these sensors with a reference position/location in space sufficiently discloses the claim element and that Watanabe’s LiDAR is not used to create an image-position model that is “determined by calibrating the at least one object detection sensor with a three-dimensional mapping provided by a spatial sensor” as claimed.
In response, the image processing field is complex having a high level of skill; as such, one of ordinary skill in the art would understand that Watanabe’s references to using LiDAR and camera data together include or at the very least motivate calibrating these data sets. Moreover, such routine and conventional details such as LiDAR-camera calibration and the related “image-position model” need not be explicitly disclosed.
MatLab is compelling evidence on the details of how to use LiDAR with a camera to create an image-position model that is “determined by calibrating the at least one object detection sensor with a three-dimensional mapping provided by a spatial sensor” and demonstrates that this claim element is highly conventional and widely known. Indeed, the MatLab toolkit includes pre-packaged software applications (apps) including Camera Calibrator app, Lidar Calibrator app, and estimateLidarCameraTransform function all of which were commercially available for widespread use by those of skill in the image processing art such that a discussion thereof need not be provided in Watanabe.
Nevertheless, a new ground of rejection that includes MatLab is made below under non-final status to give Applicant a fair opportunity to respond.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-10 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
The term “geoposition” and the claimed “determining a current geoposition data of the detected object” as well as the related use of geoposition data in the recording step of claim 1 and in the physics model of claims 2-6 has not been adequately disclosed in the instant specification in a manner that indicates the inventors had possession of the claimed invention. The plain and ordinary meaning of the term “geoposition” is the physical location of an object on the Earth typically expressed using geographical coordinates such as longitude and latitude. Further, Applicant has defined the term “geoposition” to include a geographical position including, but not limited to a global geographical position. See [0044] of the instant published application.
But there is no means for determining the position of the object on the Earth, aka, the “geoposition” of the object. For example, there is no GPS sensor that provides the latitude and longitude of any component in the disclosed system. At best, the disclosed invention determines the local 3-D position of the object relative to the camera. No data is input, sensed or determined that relates this local 3D position to a geoposition on the Earth. Applicant has also not provided a cross reference to a known techniques or otherwise admitted that there are known prior art techniques for translating local 3D position to an actual “geoposition”.
In addition, the techniques necessary for utilizing the laundry list of “spatial sensors” of claim 8 to “determin[e] a current geoposition data of the detected object using an image-position model, the image-position model determined by calibrating the at least one object detection sensor with a three-dimensional mapping provided by a spatial sensor” also not been adequately disclosed in the instant specification in a manner that indicates the inventors had possession of the claimed invention, wherein the spatial sensor is at least one of a stereo camera, a Sonar device, a RADAR device, an RF device, a device operating using photogrammetry, and a single-snap stereo fusion device
Indeed, the instant specification is limited to LiDAR and processing LiDAR data to determine local 3D position data of the detected object. As to the laundry list of other options, the instant specification engages in mere handwaving as follows
Although reference will be made herein
to LiDAR systems used to capture the spatial information, it
will be appreciated that no limitation is hereby intended to
other types of sensors useful to capture such information.
For example, data generated from other sensors can also be
used to create useful fiducials in a spatial information
dataset, including but not limited to: stereoscopic cameras,
cameras operating using photogrammetry techniques ( e.g.,
LiDAR), cameras systems using AI powered depth mapping/
estimation, Sonar, RADAR, RF methods (e.g., detecting
the location of RF transmitters), and single-snap stereo
fusion, to set just a few examples. [0028]
Quite telling is the fact that [0028] and the laundry list of claim 8 are the sole disclosure in the entire specification regarding these other “spatial sensors”. No details are provided as to how any of these sensors, besides LiDAR, may be used to detect or determine a geoposition data of the detected object. A
Also significant is the wholesale lack of disclosure regarding how an image-position model is “determined by calibrating the at least one object detection sensor with a three-dimensional mapping provided by the spatial sensor”. Merely mentioning a laundry list of alternative depth sensors without disclosing how the claimed invention may be practiced with such sensors simply does not amount to an adequate written description of the invention or of the manner and process of making and using it because it lacks full, clear, concise, and exact terms regarding the use of any depth sensor other than LiDAR and thus fails to enable any person skilled in the art to which it pertains for such other depth sensors including a stereo camera, a Sonar device, a RADAR device, an RF device, a device operating using photogrammetry, or a single-snap stereo fusion device
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 8 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 8 recites a “single-snap stereo fusion device” but this term is not understood. To what does “single-snap” refer? What is being fused?
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-10 are rejected under 35 U.S.C. 103 as being unpatentable over Watanabe (WO 2023189691 A1) and MatLab, “What Is Lidar Camera Calibration?”, 21 January 2022, downloaded from https://web.archive.org/web/20220121220626/https://www.mathworks.com/help/lidar/ug/lidar-camera-calibration.html#bu0niag and also from https://www.mathworks.com/help/lidar/ug/lidar-camera-calibration.html on 25 June 2026. It is noted that MatLab is applicable as a printed publication and under the “public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention” sections of 35 USC 102(a)(1).
Claim 1
In regards to claim 1, Watanabe discloses a non-transitory computer-readable medium storing one or more instructions that, when executed by one or more processors, are configured to cause the one or more processors to perform operations {see [0055]-[0058], [0097], fig. 26, [0153]-[0162] discussing various computer implementations including the recited, standard computer elements} comprising:
detecting an object in an image scene data to create a detected object, the image scene data representing an image scene in a field of view of at least one object detection sensor
{see person identification sub-system 104 which includes a camera (object detection sensor) and object/person detector for recognizing/detecting the object/person in image data to create a detected object including a person/object ID, Fig. 1, [0039], [0059]-[0061], [0098], [0102]};
determining a current geoposition data of the detected object using an image-position model, the image-position model determined by calibrating the at least one object detection sensor with a three-dimensional mapping provided by a spatial sensor
{The term “geoposition” is subject to a 112(a) rejection. Also the BRI of the term “geoposition” includes any 3-D position [0044].
The object tracking sub-system 106 includes, e.g., a LiDAR (spatial sensor) to determine the current 3D location data (geoposition data) of all objects within the field of view including the object/person preliminarily detected/identified by the person identification subsystem 104, [0040], [0059]-[0061]; Further as to the image position model see [0044] in which the sensors may be pre-configured (calibrated) with a reference line/plane and/or reference position/location in the space may be calculated based on known or pre-configured (calibrated) height of the object and/or focal length; [0100], [0103], [0114]. See also [0107]. The language in italics is considered highly routine and conventional such that Watanabe need not disclose but at least motivates the use thereof; see MatLab below for mapping of these highly conventional features};
determining an identity of the detected object to create a preliminary identity
{see above cites including [0059] for the person identification sub-system 104 creating an object ID that serves as a “preliminary identify” until the reliability thereof may be confirmed by various methods such as ground distance and/or speed as discussed below};
identifying a data store identity of a plurality of data store identities that matches the preliminary identity of the detected object, the plurality of data store identities corresponding to identities of a plurality of detected objects
{see above cites including [0097]-[0099], [0102] discussing a database 610 that stores various appearance data (identities) that may be used to identify a person or object}; and
recording the current geoposition data of the detected object with a data store record associated with the data store identity that matches the preliminary identity of the detected object if a physics model threshold of the detected object is satisfied
{See Fig. 5, [0094]-[0095] that determines if the detected locations are within the ground distance, [0108] compares sizes and moving speeds of detected objects, tracked object’s height or size, [0118]-[0126]; and determining if the moving speed of the object is lower than a human speed limit in [0109], [0132] to confirm reliability and assign the same object ID as per [0041], [0059], Supplementary Notes 10, 22, 23. It is noted that each of these techniques utilizes a physics model threshold, particularly the human speed limit threshold which determines if the preliminary identity of the detected object is reliable enough to confirm that the detected object’s identity and use to determine the object’s pathway. See also Figs. 7, 9, 10, 12, 13, 14 and associated disclosures}.
MatLab is a highly analogous reference from the same field of object detection and solves an analogous problem of integrating camera and LiDAR data sets using a calibration process and also demonstrates that the features disclosed therein were commercially sold and publicly available before the earliest effective filing date of the instant invention.
MatLab also teaches
detecting an object in an image scene data to create a detected object, the image scene data representing an image scene in a field of view of at least one object detection sensor
{see camera detecting an object (e.g. a building is depicted in the figure of the camera image};
determining a current geoposition data of the detected object using an image-position model, the image-position model determined by calibrating the at least one object detection sensor with a three-dimensional mapping provided by a spatial sensor
{see Lidar-camera calibration process including Lidar intrinsics, camera intrinsics and particularly extrinsic calibration that calibrates the camera sensor with a 3-D mapping provided by the LiDAR spatial sensor. Further as to the image-position model see the estimateLidarCameraTransform function}.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Watanabe which already discloses detecting an object in an image scene data to create a detected object using a camera, the image scene data representing an image scene in a field of view of at least one object detection sensor and determining a current geoposition data of the detected object such that the determination of current geoposition data includes using an image-position model, the image-position model determined by calibrating the at least one object detection sensor with a three-dimensional mapping provided by a spatial sensor as taught by MatLab because Watanabe partially discloses or at least motivates such calibration and model, because calibrating the camera and LiDAR data increases the accuracy of the system including more accurate object detection, more accurate position data and/or more accurate identification of the object, because MatLab demonstrates the highly conventional nature of such calibration and model being offered in a suite of image processing apps, because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claim 2
In regards to claim 2, Watanabe discloses wherein when the current geoposition data is recorded in the data store record with a time associated with the determination of geoposition data, wherein the data store record includes a plurality of geoposition data, each geoposition data of the plurality of geoposition data having a time associated with a determination of each of the geoposition data {See Figs. 1-3, 17, 19A illustrating this concept of recording position with time. See also [0045]-[0048] further discussing recording location and timestamp data store records}.
Claim 3
In regards to claim 3, Watanabe discloses wherein the physics model threshold is satisfied based on an evaluation of the current geoposition data with a geoposition data associated with a geoposition data in the geoposition data set that is closest in time to the current geoposition data
{See Fig. 5, [0094]-[0095] that determines if the detected locations are within the ground distance, [0108] compares sizes and moving speeds of detected objects, tracked object’s height or size, [0118]-[0126]; and determining if the moving speeds of the objects is lower than a human speed limit in [0109], [0132], Supplementary Notes 10, 22, 23. It is noted that each of these techniques utilizes a physics model threshold, particularly the human speed limit threshold which determines if the preliminary identify of the detected object is reliable enough to confirm that the detected object’s identity and use to determine the object’s pathway. See also Figs. 7, 9, 10, 12, 13, 14 and associated disclosures. Moreover, these determinations are based on evaluations of the current geoposition data with a geoposition data associated with a geoposition data in the geoposition data set that is closest in time to the current geoposition data}.
Claim 4
In regards to claim 4, Watanabe discloses wherein the physics model threshold is based on a speed of the object determined based upon a distance over which the object moves and a time difference over which the distance is measured
{See [0108] comparing sizes and moving speeds of detected objects, tracked object’s height or size, [0118]-[0126]; and determining if the moving speeds of the objects is lower than a human speed limit in [0109], [0132], Supplementary Notes 10, 22, 23. It is noted that each of these techniques utilizes a physics model threshold, particularly the human speed limit threshold which determines if the preliminary identify of the detected object is reliable enough to confirm that the detected object’s identity and use to determine the object’s pathway. Moreover, the speed is also based on the fundamental definition thereof (determined based upon a distance over which the object moves and a time difference over which the distance is measured).}.
Claim 5
In regards to claim 5, Watanabe discloses wherein the physics model threshold is at least one of a height of the object, a weight of the object, a trajectory of the object, a speed of the object, or a momentum of the object {see above cites for claim 1 which include at least height, trajectory (pathway), and/or speed of the object. Moreover, body size and body ratio are also detected in [0098] thus at least suggesting weight and momentum. Note also that this claim only requires one of the listed options}.
Claim 6
In regards to claim 6, Watanabe discloses
wherein the geoposition data is determined based upon a first object detection sensor of the at least one object detection sensor
{see person identification sub-system 104 which includes a camera (object detection sensor) and object/person detector for recognizing/detecting the object/person in image data to create a detected object including a person/object ID and geoposition data, Fig. 1, [0039], [0043]-[0044], [0059]-[0061], [0098], [0102] wherein [0100] further clarifies that location data may also be derived from image capturing device 602. Further as to plural/second object detection sensors see also Fig. 4, sensors 442A…442N, [0068]-[0075], [0093]-[0094]}; and
wherein the data store record includes geoposition data determined based upon a second object detection sensor.
{see object tracking sub-system 106 that includes, e.g., a LiDAR (spatial sensor) to determine the current 3D location data (geoposition data) of all objects within the field of view including the object/person preliminarily detected/identified by the person identification subsystem 104, [0040], [0059]-[0061]. Further as to plural/second object detection sensors see also Fig. 4, sensors 442A…442N, [0068]-[0075], [0093]-[0094]}
Claim 7
In regards to claim 7, Watanabe discloses which further includes recording a traveler detection label with the data store record, the traveler detection label indicative of whether the physics model threshold of the detected object is satisfied {see the same object ID in [0041], [0059] and object/person detection reliability determinations that record a traveler detection label (e.g. same object ID) indicative of when the physics model threshold is satisfied. See also joining of the pathways in, for example, [0109] which joints pathways when the moving speed of the object is lower than a human moving speed limit such that the joined pathways indicates or otherwise records a “traveler detection label” indicative of whether the physics model threshold (speed less than limit) of the detected object is satisfied}.
Claim 8
In regards to claim 8, Watanabe discloses wherein the spatial sensor is at least one of a stereo camera, a Sonar device, a RADAR device, an RF device, a LiDAR device, a device operating using photogrammetry, and a single-snap stereo fusion device {see LiDAR sensor [0004], [0040], [0099], [0140]}.
Claims 9 and 10
In regards to claim 9, Watanabe discloses (claim 9) which further includes determining a physical property data of the detected object based on the image scene data and the image-position model and (claim 10) wherein the physical property data including a plurality of physical property characteristics of the object including at least one of weight, height, trajectory, speed, or momentum
{see above cites for claim 1 which include at least height, trajectory (pathway), and/or speed of the object. Moreover, body size and body ratio are also detected in [0098] thus at least suggesting weight and momentum. Note also that claim 10 only requires one of the listed options. Further as to the image position model see [0044] in which the sensors may be pre-configured (calibrated) with a reference line/plane and/or reference position/location in the space may be calculated based on known or pre-configured (calibrated) height of the object and/or focal length; [0100], [0103], [0114]}.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Pelletier US 20220004768 A1 discloses tracking objects over time and employs a physics model in the form of maximum speed test for the objects to determine whether it is impossible for a person of interest to be present in the tacklet. See fig 1 copied below.
PNG
media_image1.png
744
594
media_image1.png
Greyscale
Min US 20230394686 A1 discloses object tracking in which a spatial locus of uncertainty can for example be dependent upon the detection and/or identification of the object. For example, a certain class of objects could have a maximum speed Vd_max, then the locus of uncertainty would be Vd_max multiplied by the time interval. For example, a particular identified object could have a maximum speed Vi_max, then the locus of uncertainty would be Vi_max multiplied by the time interval as per [0115]-[0119].
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Michael R Cammarata whose telephone number is (571)272-0113. The examiner can normally be reached M-Th 7am-5pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached at 571-272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL ROBERT CAMMARATA/ Primary Examiner, Art Unit 2667
Prosecution Insights