DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on October 9, 2024 and May 13, 2026 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Objections
Claims 1, 8 and 15 are objected to because of the following informalities: s
Claim 1, line 3, “instructions to;” should read “instructions to:”
Similar issue in claim 15
Claim 1, line 7, “perform recognition processing of recognizing the gaze target” should read “perform recognition processing to recognize the gaze target”
Similar issue in claims 8 and 15
Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
General Indefiniteness:
Where applicant acts as his or her own lexicographer to specifically define a term of a claim contrary to its ordinary meaning, the written description must clearly redefine the claim term and set forth the uncommon definition so as to put one reasonably skilled in the art on notice that the applicant intended to so redefine that claim term. Process Control Corp. v. HydReclaim Corp., 190 F.3d 1350, 1357, 52 USPQ2d 1029, 1033 (Fed. Cir. 1999). The terms “gaze region” and “gaze target” in claims 1-20 are used by the claim to mean “a region,” while the accepted meaning is typically “a region viewed by a user.” The term is indefinite because the specification does not clearly redefine the term. There is no human viewer, eye or act of gazing recited or defined, so it is unclear what gaze is referenced to. For the sake of examination, the examiner will interpret “gaze region” and “gaze target” as “region of interest” and “target region of interest.”
Antecedent Basis:
Claim 1 recites the limitation "the image quality control means" in line 13. There is insufficient antecedent basis for this limitation in the claim. The examiner notes it appears this should be deleted, as it appears each of the other corresponding means have been removed from the claims.
Claims 2-7 are rejected for inheriting the deficiency, while also failing to cure the deficiency noted above in claim 1.
Claim 15 recites the limitation "the image quality control means" in line 13. There is insufficient antecedent basis for this limitation in the claim. The examiner notes it appears this should be deleted, as it appears each of the other corresponding means have been removed from the claims.
Claims 16-20 are rejected for inheriting the deficiency, while also failing to cure the deficiency noted above in claim 20.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-2, 6, 8-9, 13 and 15-16 are rejected under 35 U.S.C. 103 as being unpatentable over Iqbal, Odrika, et al. "Adaptive subsampling for ROI-based visual tracking: Algorithms and FPGA implementation." IEEE Access 10 (2022): 90507-90522. (hereinafter Iqbal), and further in view of Galteri, Leonardo, et al. "Video compression for object detection algorithms." Proc. of International Conference on Pattern Recognition (ICPR). IEEE, 2018. (hereinafter Galteri).
Regarding independent claim 1, Iqbal discloses A video processing system (Figure 9, “system diagram;” abstract, “Our adaptive subsampling algorithms comprise an object detector and an ROI predictor (Kalman filter) which operate in conjunction to optimize the energy efficiency of the vision pipeline with the end task being object tracking. To further facilitate the implementation of our adaptive algorithms in real systems, we select a candidate algorithm and map it onto an FPGA;” page 90510, left column, “we give a detailed description of the video subsampling and ROI prediction problem”) comprising:
a memory configured to store instructions (Figure 9, “processing system” is read as containing a memory to execute the network), and
a processor configured to execute the instructions (Figure 9, “processing system”… “Our application runs on the processing system”) to;
perform recognition processing of recognizing the gaze target on the video (abstract, “Our adaptive subsampling algorithms comprise an object detector and an ROI predictor (Kalman filter) which operate in conjunction to optimize the energy efficiency of the vision pipeline with the end task being object tracking.”);
predict a position of the gaze target in a video subsequent to the video on which the recognition processing has been performed, based on extraction information extracted from the recognition processing (abstract, “In this work, we study how ROI programmability can be leveraged for vision applications by anticipating where the ROI will be located in future frames and switching pixels off outside of this region.”); and
determine the gaze region for which the image quality control means controls an image quality in the subsequent video, based on the predicted position of the gaze target (abstract, “In this work, we study how ROI programmability can be leveraged for vision applications by anticipating where the ROI will be located in future frames and switching pixels off outside of this region.;” image quality is maintained for the object, not the region that is not the object).
Iqbal fails to explicitly disclose as further recited. However, Galteri discloses control an image quality of a gaze region including a gaze target in an input video (page 2, right column, “The goal of the proposed approach is to learn a saliency map that can drive compression of video frames in a way that is friendly for computer vision algorithms.” … “This means that the map should indicate which part of the frame contains an object of interest for the algorithm. A secondary goal is to preserve visual quality, in terms of human visual system, for these objects;” a gaze region is read as a target region (i.e. containing an object of interest);
perform recognition processing of recognizing the gaze target on the video in which the image quality of the gaze region is controlled (page 2, right column, “The goal of the proposed approach is to learn a saliency map that can drive compression of video frames in a way that is friendly for computer vision algorithms.” … “This means that the map should indicate which part of the frame contains an object of interest for the algorithm. A secondary goal is to preserve visual quality, in terms of human visual system, for these objects;” a gaze region is read as a target region (i.e. containing an object of interest);
determine the gaze region for which the image quality control means controls an image quality in the subsequent video, based on the predicted position of the gaze target (page 2, right column, “The goal of the proposed approach is to learn a saliency map that can drive compression of video frames in a way that is friendly for computer vision algorithms.” … “This means that the map should indicate which part of the frame contains an object of interest for the algorithm. A secondary goal is to preserve visual quality, in terms of human visual system, for these objects”)
Iqbal is directed toward, “We refer to this process of ROI prediction and corresponding sensor configuration as adaptive subsampling. Our adaptive subsampling algorithms comprise an object detector and an ROI predictor (Kalman filter) which operate in conjunction to optimize the energy efficiency of the vision pipeline with the end task being object tracking (abstract).” Galteri is directed toward, “This paper describes an adaptive video coding approach for computer vision-based systems. We show how to control the quality of video compression so that automatic object detectors can still process the resulting video, improving their detection performance, by preserving the elements of the scene that are more likely to contain meaningful content (abstract).” As can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Iqbal and Galteri are directed toward similar methods of endeavor of object processing. Further, one of ordinary skill in the art before the effective filing date would be well aware that often image and video data is compressed for transporting, and downstream processing. Galteri allows for only keeping video quality good enough for a task to be completed (abstract, “We show how to control the quality of video compression so that automatic object detectors can still process the resulting video, improving their detection performance, by preserving the elements of the scene that are more likely to contain meaningful content.” page 2, right column, “If the coded video has to be principally consumed by a machine we only need to keep the video quality good enough for the task at hand to be completed successfully.”). Galteri allows the compression process to be computationally efficient, only maintaining data that could be used in downstream processing. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Galteri in order to make compression and downstream processes as efficient as possible, while also maintaining all data needed for accuracy of downstream processes.
Regarding dependent claim 2, the rejection of claim 1 is incorporated herein. Additionally, Iqbal in the combination further discloses wherein the extraction information includes time-series position information of the gaze target (abstract, “In this work, we study how ROI programmability can be leveraged for vision applications by anticipating where the ROI will be located in future frames and switching pixels off outside of this region;” prediction in future series is read as one must know the time-series position information).
Regarding dependent claim 6, the rejection of claim 1 is incorporated herein. Additionally, Iqbal in the combination further discloses wherein the processor is further configured to execute the instructions to detect an object from a video input after the video on which the recognition processing has been performed (abstract, “Our adaptive subsampling algorithms comprise an object detector and an ROI predictor (Kalman filter) which operate in conjunction to optimize the energy efficiency of the vision pipeline with the end task being object tracking.;” page 90509, left column, “In our proposed adaptive sub sampling pipeline, neural network features are exploited for anticipating future object trajectories and the Kalman filter is used for positional tracking of the objects of interest.”), and
determine the gaze region based on a matching result between the gaze target having the predicted position and the detected object (page 90509, left column, “In our proposed adaptive sub sampling pipeline, neural network features are exploited for anticipating future object trajectories and the Kalman filter is used for positional tracking of the objects of interest.”).
Regarding independent claim 8, the rejection of claim 1 applies directly. Additionally, Iqbal further discloses A video processing method (abstract, “Our adaptive subsampling algorithms comprise an object detector and an ROI predictor (Kalman filter) which operate in conjunction to optimize the energy efficiency of the vision pipeline with the end task being object tracking. To further facilitate the implementation of our adaptive algorithms in real systems, we select a candidate algorithm and map it onto an FPGA;” page 90510, left column, “we give a detailed description of the video subsampling and ROI prediction problem”) comprising:
performing recognition processing of recognizing the gaze target on a video (abstract, “Our adaptive subsampling algorithms comprise an object detector and an ROI predictor (Kalman filter) which operate in conjunction to optimize the energy efficiency of the vision pipeline with the end task being object tracking.”)
predicting a position of the gaze target in a video subsequent to the video on
which the recognition processing has been performed, based on extraction information extracted from the recognition processing (abstract, “In this work, we study how ROI programmability can be leveraged for vision applications by anticipating where the ROI will be located in future frames and switching pixels off outside of this region.”); and
determining the gaze region for which an image quality is controlled in the subsequent video, based on the predicted position of the gaze target (abstract, “In this work, we study how ROI programmability can be leveraged for vision applications by anticipating where the ROI will be located in future frames and switching pixels off outside of this region.;” image quality is maintained for the object, not the region that is not the object).
Iqbal fails to explicitly disclose as further recited. However, Galteri discloses controlling an image quality of a gaze region including a gaze target in an input video (page 2, right column, “The goal of the proposed approach is to learn a saliency map that can drive compression of video frames in a way that is friendly for computer vision algorithms.” … “This means that the map should indicate which part of the frame contains an object of interest for the algorithm. A secondary goal is to preserve visual quality, in terms of human visual system, for these objects;” a gaze region is read as a target region (i.e. containing an object of interest);
performing recognition processing of recognizing the gaze target on a video in which the image quality of the gaze region is controlled (page 2, right column, “The goal of the proposed approach is to learn a saliency map that can drive compression of video frames in a way that is friendly for computer vision algorithms.” … “This means that the map should indicate which part of the frame contains an object of interest for the algorithm. A secondary goal is to preserve visual quality, in terms of human visual system, for these objects;” a gaze region is read as a target region (i.e. containing an object of interest);
determining the gaze region for which an image quality is controlled in the subsequent video, based on the predicted position of the gaze target (page 2, right column, “The goal of the proposed approach is to learn a saliency map that can drive compression of video frames in a way that is friendly for computer vision algorithms.” … “This means that the map should indicate which part of the frame contains an object of interest for the algorithm. A secondary goal is to preserve visual quality, in terms of human visual system, for these objects”).
Iqbal is directed toward, “We refer to this process of ROI prediction and corresponding sensor configuration as adaptive subsampling. Our adaptive subsampling algorithms comprise an object detector and an ROI predictor (Kalman filter) which operate in conjunction to optimize the energy efficiency of the vision pipeline with the end task being object tracking (abstract).” Galteri is directed toward, “This paper describes an adaptive video coding approach for computer vision-based systems. We show how to control the quality of video compression so that automatic object detectors can still process the resulting video, improving their detection performance, by preserving the elements of the scene that are more likely to contain meaningful content (abstract).” As can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Iqbal and Galteri are directed toward similar methods of endeavor of object processing. Further, one of ordinary skill in the art before the effective filing date would be well aware that often image and video data is compressed for transporting, and downstream processing. Galteri allows for only keeping video quality good enough for a task to be completed (abstract, “We show how to control the quality of video compression so that automatic object detectors can still process the resulting video, improving their detection performance, by preserving the elements of the scene that are more likely to contain meaningful content.” page 2, right column, “If the coded video has to be principally consumed by a machine we only need to keep the video quality good enough for the task at hand to be completed successfully.”). Galteri allows the compression process to be computationally efficient, only maintaining data that could be used in downstream processing. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Galteri in order to make compression and downstream processes as efficient as possible, while also maintaining all data needed for accuracy of downstream processes.
Regarding dependent claim 9, the rejection of claim 8 is incorporated herein. Additionally, Iqbal in the combination further discloses wherein the extraction information includes time-series position information of the gaze target (abstract, “In this work, we study how ROI programmability can be leveraged for vision applications by anticipating where the ROI will be located in future frames and switching pixels off outside of this region;” prediction in future series is read as one must know the time-series position information).
Regarding dependent claim 13, the rejection of claim 8 is incorporated herein. Additionally, Iqbal in the combination further discloses further comprising:
detecting an object from a video input after the video on which the recognition processing has been performed (abstract, “Our adaptive subsampling algorithms comprise an object detector and an ROI predictor (Kalman filter) which operate in conjunction to optimize the energy efficiency of the vision pipeline with the end task being object tracking.;” page 90509, left column, “In our proposed adaptive sub sampling pipeline, neural network features are exploited for anticipating future object trajectories and the Kalman filter is used for positional tracking of the objects of interest.”); and
determining the gaze region based on a matching result between the gaze target having the predicted position and the detected object (page 90509, left column, “In our proposed adaptive sub sampling pipeline, neural network features are exploited for anticipating future object trajectories and the Kalman filter is used for positional tracking of the objects of interest.”).
Regarding independent claim 15, the rejection of claim 1 applies directly. Additionally, Iqbal discloses A video processing apparatus (Figure 9, “system diagram;” abstract, “Our adaptive subsampling algorithms comprise an object detector and an ROI predictor (Kalman filter) which operate in conjunction to optimize the energy efficiency of the vision pipeline with the end task being object tracking. To further facilitate the implementation of our adaptive algorithms in real systems, we select a candidate algorithm and map it onto an FPGA;” page 90510, left column, “we give a detailed description of the video subsampling and ROI prediction problem”) comprising:
a memory configured to store instructions (Figure 9, “processing system” is read as containing a memory to execute the network which is programmed and trained in instructions), and
a processor configured to execute the instructions to (Figure 9, “processing system” is read as containing a memory to execute the network which is programmed and trained in instructions);
perform recognition processing of recognizing the gaze target on the video(abstract, “Our adaptive subsampling algorithms comprise an object detector and an ROI predictor (Kalman filter) which operate in conjunction to optimize the energy efficiency of the vision pipeline with the end task being object tracking.”);
predict a position of the gaze target in a video subsequent to the video on which the recognition processing has been performed, based on extraction information extracted from the recognition processing (abstract, “In this work, we study how ROI programmability can be leveraged for vision applications by anticipating where the ROI will be located in future frames and switching pixels off outside of this region.”); and
determine the gaze region for which the image quality control means controls an image quality in the subsequent video, based on the predicted position of the gaze target (abstract, “In this work, we study how ROI programmability can be leveraged for vision applications by anticipating where the ROI will be located in future frames and switching pixels off outside of this region.;” image quality is maintained for the object, not the region that is not the object).
Iqbal fails to explicitly disclose as further recited. However, Galteri discloses control an image quality of a gaze region including a gaze target in an input video (page 2, right column, “The goal of the proposed approach is to learn a saliency map that can drive compression of video frames in a way that is friendly for computer vision algorithms.” … “This means that the map should indicate which part of the frame contains an object of interest for the algorithm. A secondary goal is to preserve visual quality, in terms of human visual system, for these objects;” a gaze region is read as a target region (i.e. containing an object of interest);
perform recognition processing of recognizing the gaze target on the video in which the image quality of the gaze region is controlled (page 2, right column, “The goal of the proposed approach is to learn a saliency map that can drive compression of video frames in a way that is friendly for computer vision algorithms.” … “This means that the map should indicate which part of the frame contains an object of interest for the algorithm. A secondary goal is to preserve visual quality, in terms of human visual system, for these objects;” a gaze region is read as a target region (i.e. containing an object of interest);
determine the gaze region for which the image quality control means controls an image quality in the subsequent video, based on the predicted position of the gaze target (page 2, right column, “The goal of the proposed approach is to learn a saliency map that can drive compression of video frames in a way that is friendly for computer vision algorithms.” … “This means that the map should indicate which part of the frame contains an object of interest for the algorithm. A secondary goal is to preserve visual quality, in terms of human visual system, for these objects”).
Iqbal is directed toward, “We refer to this process of ROI prediction and corresponding sensor configuration as adaptive subsampling. Our adaptive subsampling algorithms comprise an object detector and an ROI predictor (Kalman filter) which operate in conjunction to optimize the energy efficiency of the vision pipeline with the end task being object tracking (abstract).” Galteri is directed toward, “This paper describes an adaptive video coding approach for computer vision-based systems. We show how to control the quality of video compression so that automatic object detectors can still process the resulting video, improving their detection performance, by preserving the elements of the scene that are more likely to contain meaningful content (abstract).” As can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Iqbal and Galteri are directed toward similar methods of endeavor of object processing. Further, one of ordinary skill in the art before the effective filing date would be well aware that often image and video data is compressed for transporting, and downstream processing. Galteri allows for only keeping video quality good enough for a task to be completed (abstract, “We show how to control the quality of video compression so that automatic object detectors can still process the resulting video, improving their detection performance, by preserving the elements of the scene that are more likely to contain meaningful content.” page 2, right column, “If the coded video has to be principally consumed by a machine we only need to keep the video quality good enough for the task at hand to be completed successfully.”). Galteri allows the compression process to be computationally efficient, only maintaining data that could be used in downstream processing. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Galteri in order to make compression and downstream processes as efficient as possible, while also maintaining all data needed for accuracy of downstream processes.
Regarding dependent claim 16, the rejection of claim 15 is incorporated herein. Additionally, Iqbal in the combination further discloses wherein the extraction information includes time-series position information of the gaze target (abstract, “In this work, we study how ROI programmability can be leveraged for vision applications by anticipating where the ROI will be located in future frames and switching pixels off outside of this region;” prediction in future series is read as one must know the time-series position information).
Claim(s) 3-4, 7, 10-11, 14, 17-18 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Iqbal further in view of Galteri as applied to claims 1, 8 and 15 respectively above, and further in view of Xu, Bingjie, et al. "Interact as you intend: Intention-driven human-object interaction detection." IEEE Transactions on Multimedia 22.6 (2019): 1423-1432. (hereinafter Xu).
Regarding dependent claim 3, the rejection of claim 1 is incorporated herein. Additionally, Iqbal and Galteri in the combination fail to explicitly disclose wherein the extraction information includes an action recognition result for the gaze target.
However, Xu discloses wherein the extraction information includes an action recognition result for the gaze target (page 1, left column, “The task of HOI understanding [11]–[15] is formulated as identifying the ⟨human,action,object⟩ triplets. It is a facet of visual relationships critically driven by humans. In contrast to general visual relationships involving verbs, prepositional, spatial, and comparative phrases, HOI understanding focuses on direct interactions (actions) performed on objects (e.g. a person is holding a cup in Fig. 1).”)
As noted above, Iqbal and Galteri are directed toward similar methods of object analysis in imaging and video data. Further, Xu is directed toward “detecting human-object interactions (HOIs) in social scene images (abstract).” As can be easily seen by one of ordinary skill int eh art before the effective filing date of the claimed invention, Iqbal, Galteri and Xu are directed toward similar methods of endeavor of object analysis in imaging and video data. Further, one of ordinary skill in the art would easily understand detecting the object and a person within an image is not most helpful. The user may be more interested in the actions as related to the human and the object in the image to provide more context. Xu allows for, “The task of HOI understanding [11]–[15] is formulated as identifying the ⟨human,action,object⟩ triplets. It is a facet of visual relationships critically driven by humans. In contrast to general visual relationships involving verbs, prepositional, spatial, and comparative phrases, HOI understanding focuses on direct interactions (actions) performed on objects (e.g. a person is holding a cup in Fig. 1) (page 1, left column).” Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date to incorporate the teaching of Xu in order to gain additional understanding as related to the actions occurring within a scene, as opposed to only the objects and humans present.
Regarding dependent claim 4, the rejection of claim 3 is incorporated herein. Additionally, Xu in the combination further discloses wherein the processor is further configured to execute the instructions to predict the position of the gaze target based on a use object that is an object used in an action indicated by the action recognition result (page 1, left column, “The task of HOI understanding [11]–[15] is formulated as identifying the ⟨human,action,object⟩ triplets. It is a facet of visual relationships critically driven by humans. In contrast to general visual relationships involving verbs, prepositional, spatial, and comparative phrases, HOI understanding focuses on direct interactions (actions) performed on objects (e.g. a person is holding a cup in Fig. 1);” page 3, left column, “First, given an input image I, we adopt Faster R-CNN [2] from Detectron [40] to detect all humans and objects, generating a set of detected bounding boxes b=(b1,…,bm) where m denotes the total number of detected instances. The detected bounding boxes for a person and an object are denoted as bh and bo, respectively. ”).
One of ordinary skill in the art before the effective filing date of the claimed invention would easily be aware object position an change across video data, especially when the object is acted upon by a user (i.e. throwing a ball, picking up a cup, etc). Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Xu in order to ensure predict where the target object will appear throughout the series of video data.
Regarding dependent claim 7, the rejection of claim 1 is incorporated herein. Additionally, Iqbal and Galteri in the combination fail to explicitly disclose wherein the gaze target includes a person who is a target of the recognition processing and a use object used by the person, and the gaze region includes a region of the person and a region of the use object.
However, Xu discloses wherein the gaze target includes a person who is a target of the recognition processing and a use object used by the person (page 1, left column, “The task of HOI understanding [11]–[15] is formulated as identifying the ⟨human,action,object⟩ triplets. It is a facet of visual relationships critically driven by humans. In contrast to general visual relationships involving verbs, prepositional, spatial, and comparative phrases, HOI understanding focuses on direct interactions (actions) performed on objects (e.g. a person is holding a cup in Fig. 1);”), and
the gaze region includes a region of the person and a region of the use object (page 3, left column, “First, given an input image I, we adopt Faster R-CNN [2] from Detectron [40] to detect all humans and objects, generating a set of detected bounding boxes b=(b1,…,bm) where m denotes the total number of detected instances. The detected bounding boxes for a person and an object are denoted as bh and bo, respectively. ”).
As noted above, Iqbal and Galteri are directed toward similar methods of object analysis in imaging and video data. Further, Xu is directed toward “detecting human-object interactions (HOIs) in social scene images (abstract).” As can be easily seen by one of ordinary skill int eh art before the effective filing date of the claimed invention, Iqbal, Galteri and Xu are directed toward similar methods of endeavor of object analysis in imaging and video data. Further, one of ordinary skill in the art would easily understand a user may be interested in the different areas of data that are the object or the human, as opposed to only knowing regions which are “entities.” Xu allows for, “The task of HOI understanding [11]–[15] is formulated as identifying the ⟨human,action,object⟩ triplets. It is a facet of visual relationships critically driven by humans. In contrast to general visual relationships involving verbs, prepositional, spatial, and comparative phrases, HOI understanding focuses on direct interactions (actions) performed on objects (e.g. a person is holding a cup in Fig. 1) (page 1, left column).” Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date to incorporate the teaching of Xu in order to gain additional understanding of entities within a data set as either an image or a person to understand contexts and relationships between the two.
Regarding dependent claim 10, the rejection of claim 8 is incorporated herein. Additionally, Iqbal and Galteri in the combination fail to explicitly disclose wherein the extraction information includes an action recognition result for the gaze target.
However, Xu discloses wherein the extraction information includes an action recognition result for the gaze target (page 1, left column, “The task of HOI understanding [11]–[15] is formulated as identifying the ⟨human,action,object⟩ triplets. It is a facet of visual relationships critically driven by humans. In contrast to general visual relationships involving verbs, prepositional, spatial, and comparative phrases, HOI understanding focuses on direct interactions (actions) performed on objects (e.g. a person is holding a cup in Fig. 1).”)
As noted above, Iqbal and Galteri are directed toward similar methods of object analysis in imaging and video data. Further, Xu is directed toward “detecting human-object interactions (HOIs) in social scene images (abstract).” As can be easily seen by one of ordinary skill int eh art before the effective filing date of the claimed invention, Iqbal, Galteri and Xu are directed toward similar methods of endeavor of object analysis in imaging and video data. Further, one of ordinary skill in the art would easily understand detecting the object and a person within an image is not most helpful. The user may be more interested in the actions as related to the human and the object in the image to provide more context. Xu allows for, “The task of HOI understanding [11]–[15] is formulated as identifying the ⟨human,action,object⟩ triplets. It is a facet of visual relationships critically driven by humans. In contrast to general visual relationships involving verbs, prepositional, spatial, and comparative phrases, HOI understanding focuses on direct interactions (actions) performed on objects (e.g. a person is holding a cup in Fig. 1) (page 1, left column).” Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date to incorporate the teaching of Xu in order to gain additional understanding as related to the actions occurring within a scene, as opposed to only the objects and humans present.
Regarding dependent claim 11, the rejection of claim 10 is incorporated herein. Additionally, Xu in the combination further discloses wherein the position of the gaze target is predicted based on a use object that is an object used in an action indicated by the action recognition result (page 1, left column, “The task of HOI understanding [11]–[15] is formulated as identifying the ⟨human,action,object⟩ triplets. It is a facet of visual relationships critically driven by humans. In contrast to general visual relationships involving verbs, prepositional, spatial, and comparative phrases, HOI understanding focuses on direct interactions (actions) performed on objects (e.g. a person is holding a cup in Fig. 1);” page 3, left column, “First, given an input image I, we adopt Faster R-CNN [2] from Detectron [40] to detect all humans and objects, generating a set of detected bounding boxes b=(b1,…,bm) where m denotes the total number of detected instances. The detected bounding boxes for a person and an object are denoted as bh and bo, respectively. ”).
One of ordinary skill in the art before the effective filing date of the claimed invention would easily be aware object position an change across video data, especially when the object is acted upon by a user (i.e. throwing a ball, picking up a cup, etc). Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Xu in order to ensure predict where the target object will appear throughout the series of video data.
Regarding dependent claim 14, the rejection of claim 8 is incorporated herein. Additionally, Iqbal and Galteri in the combination fail to explicitly disclose wherein the gaze target includes a person who is a target of the recognition processing and a use object used by the person, and the gaze region includes a region of the person and a region of the use object.
However, Xu discloses wherein the gaze target includes a person who is a target of the recognition processing and a use object used by the person (page 1, left column, “The task of HOI understanding [11]–[15] is formulated as identifying the ⟨human,action,object⟩ triplets. It is a facet of visual relationships critically driven by humans. In contrast to general visual relationships involving verbs, prepositional, spatial, and comparative phrases, HOI understanding focuses on direct interactions (actions) performed on objects (e.g. a person is holding a cup in Fig. 1);”), and
the gaze region includes a region of the person and a region of the use object (page 3, left column, “First, given an input image I, we adopt Faster R-CNN [2] from Detectron [40] to detect all humans and objects, generating a set of detected bounding boxes b=(b1,…,bm) where m denotes the total number of detected instances. The detected bounding boxes for a person and an object are denoted as bh and bo, respectively. ”).
As noted above, Iqbal and Galteri are directed toward similar methods of object analysis in imaging and video data. Further, Xu is directed toward “detecting human-object interactions (HOIs) in social scene images (abstract).” As can be easily seen by one of ordinary skill int eh art before the effective filing date of the claimed invention, Iqbal, Galteri and Xu are directed toward similar methods of endeavor of object analysis in imaging and video data. Further, one of ordinary skill in the art would easily understand a user may be interested in the different areas of data that are the object or the human, as opposed to only knowing regions which are “entities.” Xu allows for, “The task of HOI understanding [11]–[15] is formulated as identifying the ⟨human,action,object⟩ triplets. It is a facet of visual relationships critically driven by humans. In contrast to general visual relationships involving verbs, prepositional, spatial, and comparative phrases, HOI understanding focuses on direct interactions (actions) performed on objects (e.g. a person is holding a cup in Fig. 1) (page 1, left column).” Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date to incorporate the teaching of Xu in order to gain additional understanding of entities within a data set as either an image or a person to understand contexts and relationships between the two.
Regarding dependent claim 17, the rejection of claim 15 is incorporated herein. Additionally, Iqbal and Galteri in the combination fail to explicitly disclose wherein the extraction information includes an action recognition result for the gaze target.
However, Xu discloses wherein the extraction information includes an action recognition result for the gaze target (page 1, left column, “The task of HOI understanding [11]–[15] is formulated as identifying the ⟨human,action,object⟩ triplets. It is a facet of visual relationships critically driven by humans. In contrast to general visual relationships involving verbs, prepositional, spatial, and comparative phrases, HOI understanding focuses on direct interactions (actions) performed on objects (e.g. a person is holding a cup in Fig. 1).”)
As noted above, Iqbal and Galteri are directed toward similar methods of object analysis in imaging and video data. Further, Xu is directed toward “detecting human-object interactions (HOIs) in social scene images (abstract).” As can be easily seen by one of ordinary skill int eh art before the effective filing date of the claimed invention, Iqbal, Galteri and Xu are directed toward similar methods of endeavor of object analysis in imaging and video data. Further, one of ordinary skill in the art would easily understand detecting the object and a person within an image is not most helpful. The user may be more interested in the actions as related to the human and the object in the image to provide more context. Xu allows for, “The task of HOI understanding [11]–[15] is formulated as identifying the ⟨human,action,object⟩ triplets. It is a facet of visual relationships critically driven by humans. In contrast to general visual relationships involving verbs, prepositional, spatial, and comparative phrases, HOI understanding focuses on direct interactions (actions) performed on objects (e.g. a person is holding a cup in Fig. 1) (page 1, left column).” Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date to incorporate the teaching of Xu in order to gain additional understanding as related to the actions occurring within a scene, as opposed to only the objects and humans present.
Regarding dependent claim 18, the rejection of claim 17 is incorporated herein. Additionally, Xu in the combination further discloses the processor is further configured to execute the instructions to predict the position of the gaze target based on a use object that is an object used in an action indicated by the action recognition result (page 1, left column, “The task of HOI understanding [11]–[15] is formulated as identifying the ⟨human,action,object⟩ triplets. It is a facet of visual relationships critically driven by humans. In contrast to general visual relationships involving verbs, prepositional, spatial, and comparative phrases, HOI understanding focuses on direct interactions (actions) performed on objects (e.g. a person is holding a cup in Fig. 1);” page 3, left column, “First, given an input image I, we adopt Faster R-CNN [2] from Detectron [40] to detect all humans and objects, generating a set of detected bounding boxes b=(b1,…,bm) where m denotes the total number of detected instances. The detected bounding boxes for a person and an object are denoted as bh and bo, respectively. ”).
One of ordinary skill in the art before the effective filing date of the claimed invention would easily be aware object position an change across video data, especially when the object is acted upon by a user (i.e. throwing a ball, picking up a cup, etc). Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Xu in order to ensure predict where the target object will appear throughout the series of video data.
Regarding dependent claim 20, the rejection of claim 15 is incorporated herein. Additionally, Iqbal and Galteri in the combination fail to explicitly disclose wherein the gaze target includes a person who is a target of the recognition processing and a use object used by the person, and the gaze region includes a region of the person and a region of the use object.
However, Xu discloses wherein the gaze target includes a person who is a target of the recognition processing and a use object used by the person (page 1, left column, “The task of HOI understanding [11]–[15] is formulated as identifying the ⟨human,action,object⟩ triplets. It is a facet of visual relationships critically driven by humans. In contrast to general visual relationships involving verbs, prepositional, spatial, and comparative phrases, HOI understanding focuses on direct interactions (actions) performed on objects (e.g. a person is holding a cup in Fig. 1);”), and
the gaze region includes a region of the person and a region of the use object (page 3, left column, “First, given an input image I, we adopt Faster R-CNN [2] from Detectron [40] to detect all humans and objects, generating a set of detected bounding boxes b=(b1,…,bm) where m denotes the total number of detected instances. The detected bounding boxes for a person and an object are denoted as bh and bo, respectively. ”).
As noted above, Iqbal and Galteri are directed toward similar methods of object analysis in imaging and video data. Further, Xu is directed toward “detecting human-object interactions (HOIs) in social scene images (abstract).” As can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Iqbal, Galteri and Xu are directed toward similar methods of endeavor of object analysis in imaging and video data. Further, one of ordinary skill in the art would easily understand a user may be interested in the different areas of data that are the object or the human, as opposed to only knowing regions which are “entities.” Xu allows for, “The task of HOI understanding [11]–[15] is formulated as identifying the ⟨human,action,object⟩ triplets. It is a facet of visual relationships critically driven by humans. In contrast to general visual relationships involving verbs, prepositional, spatial, and comparative phrases, HOI understanding focuses on direct interactions (actions) performed on objects (e.g. a person is holding a cup in Fig. 1) (page 1, left column).” Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date to incorporate the teaching of Xu in order to gain additional understanding of entities within a data set as either an image or a person to understand contexts and relationships between the two.
Claim(s) 5, 12 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Iqbal, Galteri and Xu as applied to claim 3, 10 and 17 respectively above, and further in view of Mínguez, Raúl Quintero, et al. "Pedestrian path, pose, and intention prediction through gaussian process dynamical models and pedestrian activity recognition." IEEE Transactions on Intelligent Transportation Systems 20.5 (2018): 1803-1814. (hereinafter Minguez).
Regarding dependent claim 5, the rejection of claim 3 is incorporated herein. Additionally, Iqbal, Galteri and Xu in the combination fail to explicitly disclose wherein the processor is further configured to execute the instructions to predict the position of the gaze target based on an orientation of a person who performs an action indicated by the action recognition result.
However, Minguez discloses wherein the processor is further configured to execute the instructions to predict the position of the gaze target based on an orientation of a person who performs an action indicated by the action recognition result (page 1804, right column, “Moreover, the orientations in which pedestrians are facing and head poses could be evaluated to predict future pedestrians positions.”).
As noted above, Iqbal, Galteri and Xu are directed toward object analysis in imaging and video data. Further, Minguez is directed toward, “a method to predict future pedestrian paths, poses, and intentions up to 1 s in advance (abstract).” As can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Iqbal, Galteri, Xu and Minguez are directed toward similar methods of endeavor of object analysis. Further, one of ordinary skill in the art before the effective filing date of the claimed invention would easily understand objects can have different paths based on the action taken on it by a user. Beyond that, an object can have a different trajectory based on where a user is facing (ex: a ball has a different trajectory based on what angle it was kicked from). Thus, in order to best understand the scene as a whole, and the interaction between the human and the object, it would have been obvious before the effective filing date of the claimed invention to incorporate the teaching of Minguez.
Regarding dependent claim 12, the rejection of claim 10 is incorporated herein. Additionally, Iqbal, Galteri and Xu in the combination fail to explicitly disclose wherein the position of the gaze target is predicted based on an orientation of a person who performs an action indicated by the action recognition result.
However, Minguez discloses wherein the position of the gaze target is predicted based on an orientation of a person who performs an action indicated by the action recognition result (page 1804, right column, “Moreover, the orientations in which pedestrians are facing and head poses could be evaluated to predict future pedestrians positions.”).
As noted above, Iqbal, Galteri and Xu are directed toward object analysis in imaging and video data. Further, Minguez is directed toward, “a method to predict future pedestrian paths, poses, and intentions up to 1 s in advance (abstract).” As can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Iqbal, Galteri, Xu and Minguez are directed toward similar methods of endeavor of object analysis. Further, one of ordinary skill in the art before the effective filing date of the claimed invention would easily understand objects can have different paths based on the action taken on it by a user. Beyond that, an object can have a different trajectory based on where a user is facing (ex: a ball has a different trajectory based on what angle it was kicked from). Thus, in order to best understand the scene as a whole, and the interaction between the human and the object, it would have been obvious before the effective filing date of the claimed invention to incorporate the teaching of Minguez.
Regarding dependent claim 19, the rejection of claim 17 is incorporated herein. Additionally, Iqbal, Galteri and Xu in the combination fail to explicitly disclose wherein the processor is further configured to execute the instructions to predict the position of the gaze target based on an orientation of a person who performs an action indicated by the action recognition result.
However, Minguez discloses wherein the processor is further configured to execute the instructions to predict the position of the gaze target based on an orientation of a person who performs an action indicated by the action recognition result (page 1804, right column, “Moreover, the orientations in which pedestrians are facing and head poses could be evaluated to predict future pedestrians positions.”).
As noted above, Iqbal, Galteri and Xu are directed toward object analysis in imaging and video data. Further, Minguez is directed toward, “a method to predict future pedestrian paths, poses, and intentions up to 1 s in advance (abstract).” As can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Iqbal, Galteri, Xu and Minguez are directed toward similar methods of endeavor of object analysis. Further, one of ordinary skill in the art before the effective filing date of the claimed invention would easily understand objects can have different paths based on the action taken on it by a user. Beyond that, an object can have a different trajectory based on where a user is facing (ex: a ball has a different trajectory based on what angle it was kicked from). Thus, in order to best understand the scene as a whole, and the interaction between the human and the object, it would have been obvious before the effective filing date of the claimed invention to incorporate the teaching of Minguez.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
U.S. Publication No. 2018/0213247 discloses, “A method for lossless compression of video data (abstract)”
Contact
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Courtney J. Windsor whose telephone number is (571)272-3956. The examiner can normally be reached Monday - Friday 8:00 - 4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John Villecco can be reached at 571-272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/COURTNEY JOAN NELSON/Primary Examiner, Art Unit 2661
Prosecution Insights