DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed on 03/25/2026 have been fully considered but they are not persuasive. On the interview done on 03/26/2026 examiner agreed that the current amendments overcome the art, but after further search and consideration of prior art Ngo Dinh examiner has found that the claim language of independent claims is still broad enough that they read on Ngo Dinh.
The applicant argues “Ngo Dinh does not teach or suggest that the ranking indicates "whether to further process" the image, let along using "one or more machine learning models trained to determine information related to [an] event”. Examiner respectfully disagrees. As recited by applicant paragraph [0175] of Ngo Dinh describes each image to receive a quality score (may be used to rank each image) relative to an ideal image such that an in image of poor quality needs to be ignored. Paragraph [0154] shows ignoring an image by a processor because the image is of a low quality score/ranking. A processor ignoring data is a description “not further processing said data”. However the current state of the claims “the ranking indicating whether to further process” does not describe a state that does not process the image data. In view of this Ngo Dinh still rejects the broadest most reasonable interpretation of the claims. All further arguments correspond to the above rebuttal.
Please refer to the below rejection for detail.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 3-8, 10-11, 13-15 and 17 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Ngo Dinh et al. (US 2026/0011139 A1).
With respect to Claim 1, Ngo Dinh’139 shows a device (Figure 1B paragraph [0083] the device to implement figure 1A) comprising:
one or more processors (Figure 2 230 paragraph [0078]) to:
obtain image data (video) obtained using one or more image sensors (192) (paragraph [0084]), the image data representative of one or more frames (paragraphs [0079]-[0080] input video with sequenced or temporally ordered image frames, and the image frame includes an object of interest);
determine that the image data is associated with an occurrence of an event (Figure 1A module 130 figure 3C event detector 331 paragraph [0113] Event detector 331 may determine the start and stop times in an input video of an event associated with a requested feature of interest);
determine, based at least on the event, a ranking associated with the image data (Figure 3A quality network 315 paragraph [0175] determine quality scores of each image that may be used to rank each image relative to an ideal image with requested feature of interest. Figure 3C paragraph [0112] module 10 may use scores, quality values, for each image frame to generate final output score of each image. Paragraph [0117] In some embodiments, the dashboard may include quality summary details of events identified by event detector 331 in a color-coded manner. For example, the dashboard may include red, orange, and green colored buttons or other icons to identify the quality of video of a portion of a medical procedure representing an event.), the ranking indicating whether to further process the image data using one or more machine learning models trained to determine information related to the event (Paragraph [0123] describes the module 130 (which performing the ranking) includes machine learning model that is trained based on videos of the medical procedure performed (event)); and
send, to one or more computing devices that use the one or more machine learning models (Paragraph [0094] describes the computing device 200 to include the machine learning model), the image data along with an indication of the ranking (Paragraphs [0112] and [0117] the quality/ranking of the image(s) can be displayed on a dashboard or other display in a color coded manner).
With respect to Claim 3, Ngo Dinh’139 shows the device of claim 1, wherein the one or more processors are further to: obtain sensor data obtained using one or more sensors (paragraph [0084]); and detect, based at least on the sensor data, the occurrence of the event during a period of time corresponding to at least a subset of the sensor data, wherein the image data is determined to be associated with the event based at least on a detection of the occurrence of the event during the period of time corresponding to the at least the subset of the sensor data (Figure 1A module 130 figure 3C event detector 331 paragraph [0113] Event detector 331 may determine the start and stop times in an input video of an event associated with a requested feature of interest).
With respect to Claim 4, Ngo Dinh’139 shows the device of claim 1, wherein the one or more processors are further to: obtain information that associates one or more events with one or more rankings (Figure 3A quality network 315 paragraph [0175] determine quality scores of each image that may be used to rank each image relative to an ideal image with requested feature of interest. Figure 3C paragraph [0112] module 10 may use scores, quality values, for each image frame to generate final output score of each image. Paragraph [0117] In some embodiments, the dashboard may include quality summary details of events identified by event detector 331 in a color-coded manner. For example, the dashboard may include red, orange, and green colored buttons or other icons to identify the quality of video of a portion of a medical procedure representing an event.), wherein the determination that the ranking is associated with the image data comprises: identifying, based at least on the information, the event from the one or more events (Figure 1A module 130 figure 3C event detector 331 paragraph [0113] Event detector 331 may determine the start and stop times in an input video of an event associated with a requested feature of interest); and determining, based at least on the information, that the event is associated with the ranking from the one or more rankings (Paragraph [0117] In some embodiments, the dashboard may include quality summary details of events identified by event detector 331 in a color-coded manner. For example, the dashboard may include red, orange, and green colored buttons or other icons to identify the quality of video of a portion of a medical procedure representing an event.).
With respect to Claim 5, Ngo Dinh’139 shows the device of claim 1, wherein the event comprises one or more of: a depiction of at least a portion of an object in the one or more frames (paragraphs [0125]-[0126] event detector 331, frame selector 332, and object descriptor 333 are used in combination with a display dashboard that includes one or more frames of the medical procedure including listing objects by the object descriptor in the frames of the actions determined by the event detector.); a depiction of at least a portion of an object corresponding to an object type in the one or more frames; the one or more frames being associated with motion of the object; the one or more frames being associated with a velocity of the motion; a depiction of at least a portion of a number of objects above a threshold number in the one or more frames; a depiction of an amount of light above a threshold amount in the one or more frames; or the one or more frames being associated with sound detected using audio data.
With respect to Claim 6, Ngo Dinh’139 shows the device of claim 1, wherein the one or more processors are further to: obtain second image data obtained using the one or more image sensors, the second image data representative of one or more second frames (Paragraph [0079] video refers to any digital scene or area comprised of a plurality of images in sequence. The input video sequenced or temporally ordered image frames are processed.); determine that the second image data is associated with a second event (Figure 1A module 130 figure 3C event detector 331 paragraph [0113] Event detector 331 may determine the start and stop times in an input video (described above to include first and second frames) of events (plurality) associated with a request feature of interest.); determine, based at least on the second event, a second ranking associated with the second image data that is different than the ranking associated with the image data (paragraph [0124] the percentage of total time of medical procedure for a certain action may be used for calculating the quality score of the medical procedure or a portion of the medical procedure.); and send, to the one or more computing devices, the second image data along with a second indication of the second ranking (paragraph [0124] the percentage of total time of medical procedure for a certain action may be used for calculating the quality score of the medical procedure or a portion of the medical procedure. Paragraphs [0112] and [0117] the quality/ranking of the image(s) can be displayed on a dashboard (second indication) or other display in a color coded manner.).
With respect to Claim 7, Ngo Dinh’139 shows the device of claim 6, wherein: the image data represents a first plurality of frames that includes the one or more frames (Figure 10 paragraphs [0055] and [0120] describes the summary report to include one or more image frames augmented with display markings and quality scores); the ranking is associated with the first plurality of frames (paragraph [0055] the quality scores and other information correspond to the frames); the second image data represents a second plurality of frames that includes the one or more second frames (Figure 10 and paragraphs [0055] and [0120] describes more than one procedure can be depicted in the summary); and the second ranking is associated with the second plurality of frames (paragraph [0055] the quality scores and other information correspond to the frames).
With respect to Claim 8, Ngo Dinh’139 shows the device of claim 1, wherein the one or more processors are further to: process the image data using one or more encoders to generate encoded image data (Figure 4A encoder 312 paragraph [0097]); and generate an embedded ranking by at least embedding the ranking associated with the image data into at least one of a message associated with the encoded image data (paragraph [0097] Intelligent detector system 100 may process images and detect desirable characteristics related to the feature(s) of interest using encoder 312. Intelligent detector system 100 may determine the desirable characteristics based on the trained network of encoder 312 and past determinations of feature(s) of interest.) or a field associated with a bitstream of the encoded image data, wherein the encoded image data and the embedded ranking is sent to the one or more computing devices (Figure 4A encoder 312 outputs to fully connected network 411 paragraph [0131]).
With respect to Claim 10, Ngo Dinh’139 shows the device of claim 1, wherein the device is comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing one or more simulation operations; a system for performing one or more digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing one or more deep learning operations (paragraph [0047]); a system implemented using an edge device; a system implemented using a robot; a system for performing one or more generative AI operations; a system for performing operations using one or more large language models (LLMs); a system for performing one or more conversational AI operations; a system for generating synthetic data; a system for presenting at least one of virtual reality content, augmented reality content, or mixed reality content; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources.
With respect to Claim 11, rejection analogous to those presented for claim 1, are applicable.
With respect to Claim 13, rejection analogous to those presented for claim 3, are applicable.
With respect to Claim 14, rejection analogous to those presented for claim 4, are applicable.
With respect to Claim 15, rejection analogous to those presented for claim 6, are applicable.
With respect to Claim 17, rejection analogous to those presented for claim 8, are applicable.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2, 9, 12 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Ngo Dinh et al. (US 2026/0011139 A1) in view of Katouzian et al. (US 2020/0327660 A1).
With respect to Claim 2, Ngo Dinh’139 the device of claim 1, wherein the determination that the image data is associated with the event comprises: analyzing the image data (Figures 1 and 3C module 130 paragraph [0112] to analyze each frame of the images/video.) [ ]; and determining, based at least on the analyzing, that the one or more frames depict at least a portion of the occurrence of the event (Figure 1A module 130 figure 3C event detector 331 paragraph [0113] Event detector 331 may determine the start and stop times in an input video of an event associated with a requested feature of interest).
Ngo Dinh’139 does not show using one or more second machine learning models that are different than the one or more machine learning models.
Katouzian’660 shows using one or more second machine learning models that are different than the one or more machine learning models (paragraph [0030] first and second machine learning models based on resolutions of the image data).
At the time of the invention, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claim invention to modify Ngo Dinh’139 to include using one or more second machine learning models that are different than the one or more machine learning models method taught by Katouzian’660. The suggestion/motivation for doing so would have been to improve the system’s ability to be able to improve overall efficiency of the system (paragraph [0013]).
With respect to Claim 9, Ngo Dinh’139 shows the device of claim 1,
wherein the ranking causes the one or more computing devices to at least one of: perform one or more first image processing tasks using process the image data (Figure 3C timeseries analysis module 130 paragraphs [0112]-[0130] details the various types of image processing tasks (331, 332, 333, 334) performed on the image video data) [ ]; or refrain from performing one or more second image processing tasks using the image data [ ].
Ngo Dinh’139 does not show using the one or more machine learning models when the ranking includes a first ranking; using the one or more machine learning models when the ranking includes a second ranking that is different than the first ranking.
Katouzian’660 shows using the one or more machine learning models when the ranking includes a first ranking; using the one or more machine learning models when the ranking includes a second ranking that is different than the first ranking (paragraph [0030] images ranked as low resolution are analyzed by a first model and images ranked as high resolution are analyzed by a second model and not the first).
At the time of the invention, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claim invention to modify Ngo Dinh’139 to include using the one or more machine learning models when the ranking includes a first ranking; using the one or more machine learning models when the ranking includes a second ranking that is different than the first ranking method taught by Katouzian’660. The suggestion/motivation for doing so would have been to improve the system’s ability to be able to improve overall efficiency of the system (paragraph [0013]).
With respect to Claim 12, rejection analogous to those presented for claim 2, are applicable.
With respect to Claim 18, rejection analogous to those presented for claim 9, are applicable.
Claims 19-21 are rejected under 35 U.S.C. 103 as being unpatentable over Ngo Dinh et al. (US 2026/0011139 A1) in view of Haimovitch-Yogev et al. (US 2015/0319424 A1).
With respect to Claim 19, Ngo Dinh’139 shows one or more processors comprising processing circuitry to:
obtain first sensor data using one or more first sensors (Figure 1A, paragraph [0073] user device 170 may also include an imaging device (e.g., a video or digital camera) to capture video or images for processing) and second sensor data using one or more second sensors that differ from the one or more first sensors (figure 1A physician device 180 paragraph [0074] Physician device 180 may also include an imaging device (e.g., a video or digital camera) to capture video or images for processing);
determine that the first sensor data represents an occurrence of an event (Figure 1A module 130 figure 3C event detector 331 paragraph [0113] Event detector 331 may determine the start and stop times in an input video of an event associated with a requested feature of interest);
determine, based at least on the first sensor data representing the occurrence of the event, a ranking associated with [ ] (Figure 3A quality network 315 paragraph [0175] determine quality scores of each image that may be used to rank each image relative to an ideal image with requested feature of interest. Figure 3C paragraph [0112] module 10 may use scores, quality values, for each image frame to generate final output score of each image. Paragraph [0117] In some embodiments, the dashboard may include quality summary details of events identified by event detector 331 in a color-coded manner. For example, the dashboard may include red, orange, and green colored buttons or other icons to identify the quality of video of a portion of a medical procedure representing an event.); and
causing, based at least on the ranking, one or more operations to be performed (Paragraphs [0112] and [0117] the quality/ranking of the image(s) can be displayed on a dashboard or other display in a color coded manner.) [ ].
Ngo Dinh’139 does not specifically show occurrence of the event, associated with at least a portion of the second sensor data that corresponds to the first sensor data; causing, with respect to the at least the portion of the second sensor data.
Haimovitch-Yogev’424 shows show occurrence of the event, associated with at least a portion of the second sensor data that corresponds to the first sensor data; causing, with respect to the at least the portion of the second sensor data (paragraphs [0107], [0112] and [0190] capturing an event as seen from both a first camera and the second camera and overlayed onto an environment model).
At the time of the invention, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claim invention to modify Ngo Dinh’139 to include occurrence of the event, associated with at least a portion of the second sensor data that corresponds to the first sensor data method taught by Haimovitch-Yogev’424. The suggestion/motivation for doing so would have been to improve the system’s ability to be able to efficiently handling of data between the one or more cameras (paragraph [0158]).
With respect to Claim 20, the combination of Ngo Dinh’139 and Haimovitch-Yogev’424 shows the one or more processors of claim 19, wherein the one or more processors are comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing one or more simulation operations; a system for performing one or more digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing one or more deep learning operations (in Ngo Dinh’139: paragraph [0047]); a system implemented using an edge device; a system implemented using a robot; a system for performing one or more generative AI operations; a system for performing operations using one or more large language models (LLMs); a system for performing one or more conversational AI operations; a system for generating synthetic data; a system for presenting at least one of virtual reality content, augmented reality content, or mixed reality content; a system incorporating one or more virtual machines (VMs);a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources.
With respect to Claim 21, the combination of Ngo Dinh’139 and Haimovitch-Yogev’424 shows the one or more processors of claim 19, wherein the processing circuitry is further to determine that the at least the portion of the second sensor data corresponds to the first sensor data based at least on the at least the portion of the second sensor data being obtained using the one or more second sensors while the first sensor data is obtained using the one or more first sensors (in Haimovitch-Yogev’424: paragraphs [0107], [0112] and [0190] capturing an event as seen from both a first camera and the second camera and overlayed onto an environment model).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Shichman et al. (US 2018/0132011 A1): paragraph [0175] identified an event as described herein, content generation unit 335 may generate an output video content that include multiple views of an event by including, in an output video content, segments of the event as captured by a first camera (e.g., provided in a primary video clip) and further including, in the output video content, segments of the same event as captured by a second camera (e.g., obtained from an internet or online service).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to IRIANA CRUZ whose telephone number is (571)270-3246. The examiner can normally be reached 10-6.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Akwasi M. Sarpong can be reached at (571) 270-3438. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/IRIANA CRUZ/Primary Examiner, Art Unit 2681
Prosecution Insights