DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Claim amendment filed on 06/08/2026 is acknowledged. Claims 1-21 have been amended and pending.
Response to Arguments
Applicant’s arguments with respect to claims 1-21 are acknowledged and considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 5 and 11 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 5 - “the actuator-based optical registration system” lacks antecedent basis. It is not known which optical registration system the claim is referring to. The claim is indefinite.
Regarding claim 11 - “the item tags are selected from barcodes, QR codes and April tags” lacks antecedent basis. It is not known which tags the claim is referring to. The claim is indefinite.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 1-8 and 10-21 are rejected under 35 U.S.C. 103 as being unpatentable over Ganapathi et al. US 2002/0299995 in view of Schafer et al. US 2023/0222928.
Ganapathi ‘995 discloses a system (inventory drone, Fig. 1 & 2) comprising circuitry configured to perform a vision based registration task (computer algorithms and processor onboard UV, P. 005, “… system for autonomous vehicle inventory inspection and management in a warehouse. …”), the circuitry being coupled with a vision sensor (camera) and being configured to derive a quality metric (P. 0035 & 0036 detection of fiducial markers and labels in different lighting levels and distances by vision sensor at least demonstrates quality metric is being evaluated, images and speed collected by system’s cameras are also considered quality metric for controlling drone’s flight patterns), from the output of the vision sensor (camera ), the quality metric being used in a feedback loop to generate a control signal for controlling an actuator (P. 0036 & 0037, camera images and distances are being used to control flying distances of drone from shelves, i.e. varying levels of light quality is accommodated by adjusting flight path and drone position; actuator is drone propulsion components), the actuator configured to adjust relative movement between the event-based vision sensor and a scene observed by the event-based vision sensor ( P.0036, “…sensors and cameras and control methodologies, the UV is able to control its position to within a few centimeters from its prescribed path… despite … varying levels of light in the warehouse …”, i.e. control flight path to accommodate for varying levels of light to detect labels and markers).
Ganapathi ‘995 P. 0032 discloses that other types of cameras could be used on the drone. However, it is not specific concerning event-based camera.
Schafer ‘928 disclose an event-based camera mounted on a drone for capturing images of objects and scenery while in flight.
It would have been obvious for one of ordinary skill in the art before the filing date of the claimed invention to use an event-based camera on Ganapathi’s drone as another camera means for capturing images while the drone is in fight per Schafer ‘928 teaching. This modification yields expected result of providing rapid image capturing with less motion blur because HDR cameras are known for handling extreme lighting contrasts.
Regarding claim 2 - Ganapathi ‘995 P. 0032 circuitry is configured to find the state in which the system performs the vision-based registration task optimally ( “…sensors and cameras and control methodologies, the UV is able to control its position to within a few centimeters from its prescribed path…”, i.e. control flight path to accommodate for varying levels of light to detect labels and markers)..
Regarding claim 3 - Ganapathi ‘995 circuitry is configured to determine a quality metric from the output of the vision sensor, the quality metric being used in a feedback loop to control an actuator and thus affect the quality of what is being sensed by the event-based vision sensor ( P.0036 & 0037, camera image quality and distances are being used to control flying distances of drone from shelves, “… UV is able to control its position to within a few centimeters from its prescribed path… despite … varying levels of light in the warehouse …”).
Ganapathi ‘995 P. 0032 discloses that other types of cameras could be used on the drone. However, it is not specific concerning event-based camera.
Schafer et al. 2023/0222928 disclose an event-based camera mounted on a drone for capturing images of objects and scenery while in flight.
It would have been obvious for one of ordinary skill in the art before the filing date of the claimed invention to use an event-based camera on Ganapathi’s drone as another camera means for capturing images while the drone is in fight per Schafer ‘928 teaching. This modification yields expected result of providing rapid image capturing with less motion blur because HDR cameras are known for handling extreme lighting contrasts.
Regarding claim 4- Ganapathi ‘995 system itself or an object can be moved in its relative position to the system by an actuator which gets control input based on quality metric derived from the output of the vision sensor and/or the consistency of the task output (P. 0035 -P0037).
Ganapathi ‘995 P. 0032 discloses that other types of cameras could be used on the drone. However, it is not specific concerning event-based camera.
Schafer et al. 2023/0222928 disclose an event-based camera mounted on a drone for capturing images of objects and scenery while in flight.
It would have been obvious for one of ordinary skill in the art before the filing date of the claimed invention to use an event-based camera on Ganapathi’s drone as another camera means for capturing images while the drone is in fight per Schafer ‘928 teaching. This modification yields expected result of providing rapid image capturing with less motion blur because HDR cameras are known for handling extreme lighting contrasts.
Regarding claim 5- Ganapathi ‘995 optical inventory registration system comprises a sensing subsystem (any system other than the mechanical structure of the drone is interpreted to be a subsystem) which comprises the vision sensor (P. 0032, Fig. 2).
Ganapathi ‘995 P. 0032 discloses that other types of cameras could be used on the drone. However, it is not specific concerning event-based (EVS) camera.
Schafer et al. 2023/0222928 disclose an event-based camera mounted on a drone for capturing images of objects and scenery while in flight.
It would have been obvious for one of ordinary skill in the art before the filing date of the claimed invention to use an event-based camera on Ganapathi’s drone as another camera means for capturing images while the drone is in fight per Schafer ‘928 teaching. This modification yields expected result of providing rapid image capturing with less motion blur because HDR cameras are known for handling extreme lighting contrasts.
Regarding claim 6- Ganapathi ‘995 system is moved by an actuator, the actuator getting control input based on the inference quality metric derived from output of the vision sensor the quality metric derived from the output of the vision sensor (P.0035-0036)
Ganapathi ‘995 P. 0032 discloses that other types of cameras could be used on the drone. However, it is not specific concerning event-based (EVS) cameras.
Schafer et al. 2023/0222928 disclose an event-based camera mounted on a drone for capturing images of objects and scenery while in flight.
It would have been obvious for one of ordinary skill in the art before the filing date of the claimed invention to use an event-based camera on Ganapathi’s drone as another camera means for capturing images while the drone is in fight per Schafer ‘928 teaching. This modification yields expected result of providing rapid image capturing with less motion blur because HDR cameras are known for handling extreme lighting contrasts.
Regarding claim 7- Ganapathi ‘995 comprising a sensing subsystem (any system other than the mechanical structure of the drone is interpreted to be a subsystem - processors, algorithms, etc. onboard UV) which comprises the vision sensors, and wherein the sensor subsystem uses events measured by sensors to reconstruct item tags and/or navigational cues.
Ganapathi ‘995 P. 0032 discloses that other types of cameras could be used on the drone. However, it is not specific concerning event-based (EVS) cameras.
Schafer et al. 2023/0222928 disclose an event-based camera mounted on a drone for capturing images of objects and scenery while in flight.
It would have been obvious for one of ordinary skill in the art before the filing date of the claimed invention to use an event-based camera on Ganapathi’s drone as another camera means for capturing images while the drone is in fight per Schafer ‘928 teaching. This modification yields expected result of providing rapid image capturing with less motion blur because HDR cameras are known for handling extreme lighting contrasts.
Regarding claim 8- Ganapathi ‘995 EVS-based sensing subsystem (any system other than the mechanical structure of the drone is interpreted to be a subsystem) which comprises the vision sensor, and wherein the sensor subsystem is configured to output an inference quality metric (detected labels and markers in different lighting conditions and/or speed of drone, P. 0035-0036).
Ganapathi ‘995 P. 0032 discloses that other types of cameras could be used on the drone. However, it is not specific concerning event-based (EVS) cameras.
Schafer et al. 2023/0222928 disclose an event-based camera mounted on a drone for capturing images of objects and scenery while in flight.
It would have been obvious for one of ordinary skill in the art before the filing date of the claimed invention to use an event-based camera on Ganapathi’s drone as another camera means for capturing images while the drone is in fight per Schafer ‘928 teaching. This modification yields expected result of providing rapid image capturing with less motion blur because HDR cameras are known for handling extreme lighting contrasts.
Regarding claim 10- Ganapathi ‘995 EVS- sensing subsystem (any system other than the mechanical structure of the drone is interpreted to be a subsystem) is implemented onboard a scanning drone, the movement of the scanning drone /robot being adaptive to inference quality derived from output of the vision sensor based on a quality metric derived from the output of the vision sensor (Fig. 2, at least P005-0036).
Ganapathi ‘995 P. 0032 discloses that other types of cameras could be used on the drone. However, it is not specific concerning event-based (EVS) cameras.
Schafer et al. 2023/0222928 disclose an event-based camera mounted on a drone for capturing images of objects and scenery while in flight.
It would have been obvious for one of ordinary skill in the art before the filing date of the claimed invention to use an event-based camera on Ganapathi’s drone as another camera means for capturing images while the drone is in fight per Schafer ‘928 teaching. This modification yields expected result of providing rapid image capturing with less motion blur because HDR cameras are known for handling extreme lighting contrasts.
Regarding claim 11- Ganapathi ‘995 item tags are selected from barcodes, QR codes and April tags and wherein the vision-based registration task comprises a process of tag detection (P.006, 0016, etc.).
Regarding claim 12 - Ganapathi ‘995 circuitry is configured to transform an event representation of events obtained from the vision sensor (Fig. 5-7).
Regarding claim 13 - Ganapathi ‘995 circuitry is configured to perform image reconstruction based on an event stream obtained by the vision sensor (Fig. 5-7).
Ganapathi ‘995 P. 0032 discloses that other types of cameras could be used on the drone. However, it is not specific concerning event-based (EVS) cameras.
Schafer et al. 2023/0222928 disclose an event-based camera mounted on a drone for capturing images of objects and scenery while in flight.
It would have been obvious for one of ordinary skill in the art before the filing date of the claimed invention to use an event-based camera on Ganapathi’s drone as another camera means for capturing images while the drone is in fight per Schafer ‘928 teaching. This modification yields expected result of providing rapid image capturing with less motion blur because HDR cameras are known for handling extreme lighting contrasts.
Regarding claim 14 - Ganapathi ‘995 circuitry is configured to perform a specified task on reconstructed images to obtain a task output (Fig. 5-7).
Regarding claim 15 - Ganapathi ‘995 circuitry is configured to perform inference of image quality is performed based on reconstructed images to obtain an image quality metric (Fig. 5-7).
Regarding claim 16 - Ganapathi ‘995 circuitry is configured to perform a consistency check on the task output to obtain a consistent metric (maintaining consistent captured image quality and prevent degradation due to angular drift, P.0034)
Regarding claim 17 - Ganapathi ‘995 circuitry is configured to perform a control loop based on an image quality metric and/or a consistence metric (P.0034, P. 0035, P.0036).
Regarding claim 18 - Ganapathi ‘995 circuitry is configured to deduce the possibility of the current frame containing an image tag, and/or the circuitry is configured to localize a tag (Fig. 5-7).
Regarding claim 19 - Ganapathi ‘995 discloses a vision sensing subsystem comprising circuitry configured to perform a vision-based registration task (inventory registration task), the subsystem comprising a vision sensor and circuitry configured to output an inference quality metric (inference quality metric = barcode reading, item counting, location detection, etc.) to a processor of a vision-based registration task system.
Ganapathi ‘995 P. 0032 discloses that other types of cameras could be used on the drone. However, it is not specific concerning event-based (EVS) cameras.
Schafer et al. 2023/0222928 disclose an event-based camera mounted on a drone for capturing images of objects and scenery while in flight.
It would have been obvious for one of ordinary skill in the art before the filing date of the claimed invention to use an event-based camera on Ganapathi’s drone as another camera means for capturing images while the drone is in fight per Schafer ‘928 teaching. This modification yields expected result of providing rapid image capturing with less motion blur because HDR cameras are known for handling extreme lighting contrasts.
Regarding claim 20 - Ganapathi ‘995 discloses a computer-implemented method for performing a vision- based registration task (for inventory), the method comprising acquiring output of a vision sensor deriving a quality metric from the output of the vision sensor; and generating based on the quality metric, a control signal for controlling an actuator to adjust relative movement between the vision sensor and a scene observed by the event-based vision sensor ( “ … sensors and cameras and control methodologies, the UV is able to control its position to within a few centimeters from its prescribed path… despite … varying levels of light in the warehouse …”, i.e. control flight path to accommodate for varying levels of light to detect labels and markers)..
Ganapathi ‘995 P. 0032 discloses that other types of cameras could be used on the drone. However, it is not specific concerning event-based (EVS) cameras.
Schafer et al. 2023/0222928 disclose an event-based camera mounted on a drone for capturing images of objects and scenery while in flight.
It would have been obvious for one of ordinary skill in the art before the filing date of the claimed invention to use an event-based camera on Ganapathi’s drone as another camera means for capturing images while the drone is in fight per Schafer ‘928 teaching. This modification yields expected result of providing rapid image capturing with less motion blur because HDR cameras are known for handling extreme lighting contrasts.
Regarding claim 21 - Ganapathi ‘995 discloses a non-transitory computer-readable storage medium (P. 0043, “… method could include a non-transitory computer storage medium …”) storing a program comprising instructions, the instructions being configured to, when operated by a processor, perform the method of claim 20.
Claim 1 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Habibi et al. US 2007/0073439 in view of Schafer et al. US 2023/0222928.
Regarding claim 1 - Habibi ‘439 discloses a system (Fig. 1-5) comprising circuitry (300, Fig.4) configured to perform a vision based registration task (object tracking system via movable camera system 100, Fig. 1, 4, 5), the circuitry being coupled with a vision sensor (camera 120) and being configured to derive a quality metric (object edge detection, occlusion detection, speed, etc., P. 0074, 0075, 0099), from the output of the vision sensor (camera 120 ), the quality metric being used in a feedback loop to generate a control signal for controlling an actuator (P.0161, “…at least one actuator coupled to move at least one of the camera …”, the actuator configured to adjust relative movement between the event-based vision sensor and a scene observed by the event-based vision sensor ( P.0162, “… Causing at least one actuator to move at least one of the camera or the object based at least in part on the determined position of the object with respect to the camera to at least partially avoid an occlusion of a view of the object by the camera…”).
Habibi ‘439 is silent as to the specific of an event-based (EVS) camera.
Schafer ‘928 disclose an event-based camera mounted on movable means for capturing images of objects and scenery.
It would have been obvious for one of ordinary skill in the art before the filing date of the claimed invention to use an event-based camera for Habibi’s vision system as another camera means for capturing images while it is moving per Schafer ‘928 teaching. This modification yields expected result of providing rapid image capturing with less motion blur due to operating nature of HDR cameras.
Regarding claim 09 – Habibi ‘439 discloses a conveyor belt (112) which is coupled to a vision sensing subsystem (100), the movement of the conveyor belt being adaptive controlled based on a quality metric derived from the output of the vision sensor (P. 0160, “… changing a speed at which the object is translating …”)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Skaff et al. US 2022/0303445 discloses a mobile camera inventory system wherein the camera movement is based on quality metric.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Supervisory Patent Examiner Khoi H. Tran whose telephone number is (571)272-6919. The examiner can normally be reached Mon-Thurs 7:00 AM to 5:30 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KHOI H TRAN/Supervisory Patent Examiner, Art Unit 3656
