DETAILED ACTION
Claims 1-20 are pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 13 February 2025 and 3 October 2025 have been considered by the examiner.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
Claim Objections
Applicant is advised that should claims 1-7 be found allowable, claims 15-20 will be objected to under 37 CFR 1.75 as being a substantial duplicate thereof. When two claims in an application are duplicates or else are so close in content that they both cover the same thing, despite a slight difference in wording, it is proper after allowing one claim to object to the other as being a substantial duplicate of the allowed claim. See MPEP § 608.01(m).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 8, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (U.S. Publication No. US 2012/0008832 A1) ("Wang") in view of Furukawa (U.S. Publication No. US 2008/0100709 A1) ("Furukawa").
Regarding claim 1, Wang discloses a video processing system (Abstract; Fig. 1 element 10 ; [0008]) comprising:
a memory configured to store instructions ([0024-0025]), and
a processor configured to execute the instructions ([0024-0025]) to:
control image quality of a gaze region including a gaze target in an input video (Fig. 1 element 16; [0008-0009], [0011-0013], and [0017], wherein the video quality enhancement (VQE) module locally enhances the image quality (i.e. control image quality) of the region of interest (i.e. gaze region) around the detected object (i.e. gaze target) in the video (i.e. input video, where a video input is provided to the video processor 10 in Fig. 1));
perform recognition processing regarding an object included in the video in which the image quality of the gaze region is controlled (Fig. 1 elements 16 and 18; [(0019-0020], wherein the VQE module provides the enhanced ROI (i.e. controlled image quality of the gaze region) to the object recognition module for object recognition/identification (i.e. recognition processing regarding an object)); and
However, as indicated by the double strikethroughs above, Wang fails to teach extract the gaze target based on a temporal change in recognition result of the recognition processing. Furukawa, on the other hand, teaches detecting and extracting a moving object based upon a time-dependent difference between two or more consecutive images. More specifically, as it relates to the applicant’s claims, Furukawa discloses extract the gaze target based on a temporal change in recognition result of the recognition processing ([0023] and [0053], wherein the moving object is detected based upon a time-dependent difference between two or more consecutive images (i.e. temporal change in recognition result of the recognition processing) and extracting at least one mask section of reduced size covering the moving object (i.e. extract the gaze target).
Furukawa is combinable with Wang because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to more accurately identify the target moving object (Furukawa, [0003] and [0053]).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate extract the gaze target based on a temporal change in recognition result of the recognition processing, as taught by Furukawa, into the video processing system, as taught by Wang, to obtain the invention as specified in claim 1.
Claims 8 and 15 have limitations that are substantially similar to claim 1. Therefore, the rejection applied to claim 1 is also applies equally to claims 8 and 15. Furthermore, Wang teaches a video processing method (Abstract; [0008]) and a video processing apparatus ([0024-0025]).
Claims 2-4, 9-11, and 16-18 are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (U.S. Publication No. US 2012/0008832 A1) ("Wang") in view of Furukawa (U.S. Publication No. US 2008/0100709 A1) ("Furukawa") and further in view of Chi et al. (U.S. Publication No. US 2022/0051061 A1) ("Chi").
Regarding claim 2, Wang in view of Furukawa disclose the video processing system according to claim 1, (Furukawa, [0023] and [0053], wherein the moving object is detected based upon a time-dependent difference between two or more consecutive images (i.e. temporal change in recognition result of the recognition processing) and extracting at least one mask section of reduced size covering the moving object (i.e. extract the gaze target).
However, as indicated by the double strikethroughs above, Wang and Furukawa fail to teach wherein the recognition processing includes processing of recognizing an action of the object, and the processor is further configured to execute the instructions to extract the gaze target based on a temporal change in a score of an action label included in the recognition result, specifically (emphasis added). Chi, on the other hand, teaches action recognition of an object within video frames by determining temporal changes in action categories/scores. More specifically and as it relates to the applicant’s claims, Wang and Furukawa in view of Chi disclose wherein the recognition processing includes processing of recognizing an action of the object (Chi, Fig. 1 and 2; [0037], [0050], [0061-0064], wherein action recognition of an object of interest is determined) and the processor is further configured to execute the instructions to extract the gaze target based on a temporal change in a score of an action label (Chi, [0063] and [0080-0088], wherein action categories are determined (i.e. action label); [0152-0157], wherein a feature value is calculated based on action categories (i.e. score of an action label) according to the order of the video frames, the feature value accumulates to reflect the continuity of the action category in the time dimension (where feature value accumulation corresponds to temporal change in a score of the action label), and when the accumulated feature value reaches a trigger threshold, the action is determined) included in the recognition result.
Chi is combinable with Wang and Furukawa because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to reduce the possibility of misjudgment due to calculation errors of the action category or unintentional actions of the interactive object and improve the robustness of the entire system (Chi, [0150]).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate , wherein the recognition processing includes processing of recognizing an action of the object, and the processor is further configured to execute the instructions to extract the gaze target based on a temporal change in a score of an action label included in the recognition result, as taught by Chi, into the video processing system, as taught by Wang and Furukawa, to obtain the invention as specified in claim 2.
Regarding claim 3, Wang, Furukawa, and Chi disclose the video processing system according to claim 2.
Wang in view of Furukawa and further in view of Chi disclose wherein the processor is further configured to execute the instructions to extract the gaze target (Furukawa, [0023] and [0053], wherein the moving object is detected based upon a time-dependent difference between two or more consecutive images (i.e. temporal change in recognition result of the recognition processing) and extracting at least one mask section of reduced size covering the moving object (i.e. extract the gaze target)) based on a temporal increase amount of the score of the action label, specifically (emphasis added). Chi, on the other hand, teaches an accumulation of feature value as the same action category continues to appear in successive frames and an action is determined if the feature value reaches the threshold. More specifically and as it relates to the applicant’s claims, Wang and Furukawa in view of Chi discloses wherein the processor is further configured to execute the instructions to extract the gaze target based on a temporal increase amount of the score of the action label ([0153-0157], [0172-0175], wherein the feature value accumulates to reflect the continuity of the action category in the time dimension (where feature value accumulation corresponds to temporal increase in a score of the action label), and when the accumulated feature value reaches a trigger threshold (where the trigger threshold corresponds to increase amount of the score of the action label), the action is determined).
Chi is combinable with Wang and Furukawa because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to reduce the possibility of misjudgment due to calculation errors of the action category or unintentional actions of the interactive object and improve the robustness of the entire system (Chi, [0150]).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate wherein the processor is further configured to execute the instructions to extract the gaze target based on a temporal increase amount of the score of the action label, as taught by Chi, into the video processing system, as taught by Wang, Furukawa, and Chi, to obtain the invention as specified in claim 3.
Regarding claim 4, Wang, Furukawa, and Chi disclose the video processing system according to claim 2.
Wang in view of Furukawa and further in view of Chi disclose wherein the processor is further configured to execute the instructions to extract the gaze target (Furukawa, [0023] and [0053], wherein the moving object is detected based upon a time-dependent difference between two or more consecutive images (i.e. temporal change in recognition result of the recognition processing) and extracting at least one mask section of reduced size covering the moving object (i.e. extract the gaze target)) based on temporal variation in the score of the action label, specifically (emphasis added). Chi, on the other hand, teaches an accumulation of feature value as the same action category continues to appear in successive frames and an action is determined if the feature value reaches the threshold. More specifically and as it relates to the applicant’s claims, Wang and Furukawa in view of Chi disclose wherein the processor is further configured to execute the instructions to extract the gaze target based on temporal variation in the score of the action label ([0154], wherein the feature value is increased as the action category corresponding to the interactive action appears, and the feature value is reduced as the action category corresponding to the interactive action does not appear (i.e. temporal variation in the score of the action label); [0153-0157], [0172-0175], wherein the feature value accumulates to reflect the continuity of the action category in the time dimension (where feature value accumulation corresponds to temporal variation in a score of the action label), and when the accumulated feature value reaches a trigger threshold (where the trigger threshold corresponds to increase amount of the score of the action label), the action is determined)..
Chi is combinable with Wang and Furukawa because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to reduce the possibility of misjudgment due to calculation errors of the action category or unintentional actions of the interactive object and improve the robustness of the entire system (Chi, [0150]).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate wherein the processor is further configured to execute the instructions to extract the gaze target based on temporal variation in the score of the action label, as taught by Chi, into the video processing system, as taught by Wang, Furukawa, and Chi, to obtain the invention as specified in claim 4.
Claims 9-11 and 16-18 have limitations that are substantially similar to claims 2-4, respectively. Therefore, the rejections applied to claims 2-4 are also applied equally to claims 9-11 and 16-18, respectively. Furthermore, Wang teaches a video processing method (Abstract; [0008]) and a video processing apparatus ([0024-0025]).
Claims 5, 7, 12, 14, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (U.S. Publication No. US 2012/0008832 A1) ("Wang") in view of Furukawa (U.S. Publication No. US 2008/0100709 A1) ("Furukawa") and Chi et al. (U.S. Publication No. US 2022/0051061 A1) ("Chi") and further in view of Gu et al. (U.S. Publication No. US 2022/0171964 A1) ("Gu").
Regarding claim 5, Wang, Furukawa, and Chi disclose the video processing system according to claim 2.
Wang in view of Furukawa and further in view of Chi disclose wherein the processor is further configured to execute the instructions to extract the gaze target (Furukawa, [0023] and [0053], wherein the moving object is detected and extracting at least one mask section of reduced size covering the moving object (i.e. extract the gaze target))
However, as indicated by the double strikethroughs above, Wang, Furukawa, and Chi fail to teach wherein the processor is further configured to execute the instructions to extract the gaze target based on a positional relationship between a related object related to the recognized action and another object, specifically (emphasis added). Gu, on the other hand, teaches determining whether there is an interaction by determining the positional relationship between the hand and the target item. More specifically and as it relates to the applicant’s claims, Wang, Furukawa, and Chi in view of Gu disclose wherein the processor is further configured to execute the instructions to extract the gaze target based on a positional relationship between a related object related to the recognized action and another object (Gu, Fig. 2 step S205 & Fig. 3 step 2051; [0040], [0046], [0048], and [0050], wherein the positional relationship between the monitored target object/hand (i.e. related object related to the recognized action) and the target item (i.e. another object) to determine whether the item is being held or controlled).
Gu is combinable with Wang, Furukawa, and Chi because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to improve the detection efficiency and reduce the required computing resources (Gu, [0053]).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate wherein the processor is further configured to execute the instructions to extract the gaze target based on a positional relationship between a related object related to the recognized action and another object, as taught by Gu, into the image processing system, as taught by Wang, Furukawa, and Chi, to obtain the invention as specified in claim 5.
Regarding claim 7, Wang, Furukawa, Chi, and Gu disclose the video processing system according to claim 5.
However, Wang, Furukawa, and Chi fail to teach wherein the processor is further configured to execute the instructions to determine the another object as the gaze target in a case where overlap between a region including the related object and a region including the another object is larger than a predetermined threshold. Gu, on the other hand, teaches determining an area of overlapping region between the position of the hand and a position of the target item and when the overlap is over greater than a threshold, then the hand is interacting with the target item. More specifically and as it relates to the applicant’s claims, Gu discloses wherein the processor is further configured to execute the instructions to determine the another object as the gaze target in a case where overlap between a region including the related object and a region including the another object is larger than a predetermined threshold (Gu, Fig. 3. step 2051-2052; [0009], [0048], [0066], wherein the area of the overlapping region between the position of the hand (i.e. region including the related object) and the position of the target item (i.e. region including the another object) is determined and if the area of the overlapping is greater than a preset overlapping threshold (i.e. larger than a predetermined threshold), then it is determined that the hand is interacting with the detected target item (i.e. determine the another object as the gaze target)).
Gu is combinable with Wang, Furukawa, and Chi because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to reduce the range of image processing, reduce the resource consumption of the image processor, and the accuracy of the image processing result can be improved at the same time (Gu, [0077]).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate wherein the processor is further configured to execute the instructions to determine the another object as the gaze target in a case where overlap between a region including the related object and a region including the another object is larger than a predetermined threshold, as taught by Gu, into the image processing system, as taught by Wang, Furukawa, Chi, and Gu, to obtain the invention as specified in claim 7.
Claims 12 and 19 have a limitation that is substantially similar to claim 5. Therefore, the rejection applied to claim 5 is also applied equally to claims 12 and 19. Furthermore, Wang teaches a video processing method (Abstract; [0008]) and a video processing apparatus ([0024-0025]).
Claim 14 has a limitation that is substantially similar to claim 7. Therefore, the rejection applied to claim 7 is also applied equally to claim 14. Furthermore, Wang teaches a video processing method (Abstract; [0008]).
Claims 6, 13, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (U.S. Publication No. US 2012/0008832 A1) ("Wang") in view of Furukawa (U.S. Publication No. US 2008/0100709 A1) ("Furukawa"), Chi et al. (U.S. Publication No. US 2022/0051061 A1) ("Chi") and Gu et al. (U.S. Publication No. US 2022/0171964 A1) ("Gu") and further in view of Zhang et al. (U.S. Publication No. US 2023/0063926 A1) ("Zhang").
Regarding claim 6, Wang, Furukawa, Chi, and Gu disclose the image processing system according to claim 5.
However, Wang, Furukawa, Chi, and Gu fail to teach wherein the processor is further configured to execute the instructions to determine the another object as the gaze target in a case where a distance between the related object and the another object is smaller than a predetermined threshold. Zhang, on the other hand, teaches comparing the distance between a person’s wrist and a package’s bounding box to a threshold to determine that there is a holding/pickup action of the package. More specifically and as it relates to the applicant’s claims, Zhang discloses wherein the processor is further configured to execute the instructions to determine the another object as the gaze target in a case where a distance between the related object and the another object is smaller than a predetermined threshold (Fig. 5A & 5B element 506 and 508; [0102] and [0104-0106], wherein if the calculated distance between a person’s wrist (i.e. related object) and a package’s bounding box(i.e. another object) is less than the predetermined distance (i.e. predetermined threshold), then it is determined that the person is holding or picking up the detected package (i.e. determine the another object as the gaze target)).
Zhang is combinable with Wang, Furukawa, Chi, and Gu because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to speed up the runtime of downstream processes and improve the accuracy of results. (Zhang, [0088]).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate wherein the processor is further configured to execute the instructions to determine the another object as the gaze target in a case where a distance between the related object and the another object is smaller than a predetermined threshold, as taught by Zhang, into the video processing system, as taught by Wang, Furukawa, Chi, and Gu, to obtain the invention as specified in claim 6.
Claim 13 has a limitation that is substantially similar to claim 6. Therefore, the rejection applied to claim 6 is also applied equally to claim 13. Furthermore, Wang teaches a video processing method (Abstract; [0008]).
Claim 20 has limitations that are substantially similar to claim 6 and 7. Therefore, the rejections applied to claims 6 and 7 are also applied equally to claim 20. Furthermore, Wang teaches a video processing apparatus ([0024-0025]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Noorkami et al. (U.S. Publication No. US 2020/0195940 A1) teaches determining a region of interest based on a location within the frame of video associated with the object and the smoothed gaze estimate to enhance the frame of video based on the region of interest (Fig. 6 steps 630 and 650; [0061] and [0064]). Then, computer vision processing detects an object appearing within the region of interest of the video frame to determine a classification of the object (Fig. 7 step 710; [0068-0069]) and the gaze target is extracted by applying a temporal filter to the gaze data, using the recognition output for object location (Fig. 6 step 620; [0061] and [0063]).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RACHEL Y DANG whose telephone number is (571)438-9519. The examiner can normally be reached Monday - Thursday: 7:30am - 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John Villecco can be reached at (571) 272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RACHEL Y DANG/Examiner, Art Unit 2661
/JOHN VILLECCO/Supervisory Patent Examiner, Art Unit 2661
Prosecution Insights