DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgement is made of Applicant’s claim of priority from KR10-2023-0001412, filed January 4, 2023.
Information Disclosure Statement
The information disclosure statements (“IDS”) filed on January 3, 2024 were reviewed and the listed references were noted.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “a video frame acquisition unit that confirms…”, “a region of interest estimation unit that extracts…”, “a final region of interest determination unit that determines…” in claims 12-19.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 20 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter because the recitation “a computer-readable recording medium” could be directed to transitory forms of signal transmission (often referred to as "signals per se"), such as a propagating electrical or electromagnetic signal or carrier wave, which are not directed to any of the statutory categories (see MPEP 2106.03.I).
The rejection of claim 20 may be overcome by amending the claim to, for example, recite as: “A non-transitory computer-readable recording medium…”.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claims recite a system, method, and non-transitory computer-readable medium for extracting a region of interest. Consider method claim 1:
Step 1:
With regard to Step 1, the instant claim is directed to a method or a process; and therefore, the claim is directed to one of the statutory categories of invention.
Step 2A, Prong One:
With regard to 2A, Prong One, the limitations “confirming whether a region of interest of a current frame is a cluster-centered region of interest or an object-centered region of interest”, “extracting a tracking-based region of interest of the current frame according to a type of the region of interest”, “determining a final region of interest of the current frame based on the extracted tracking-based region of interest of the current frame”, “wherein, in response to the type of the region of interest being confirmed as the cluster-centered region of interest, extracting the tracking-based region of interest is performed for each cluster comprised of multiple objects in the current frame” and “wherein, in response to the type of the region of interest being confirmed as the object-centered region of interest, extracting the tracking-based region of interest is performed based on a single object” as drafted, recite an abstract idea, such as a process that, under its broadest reasonable interpretation, covers performance of the limitations manually and in the mind of a person. That is, a user or person skilled in the art may confirm whether a region of interest contains a cluster of objects or a single object, determine either a cluster of objects or a single object as the region of interest in a current frame based on the confirmation, and determine a final region of interest in the current frame. This is the concept that falls under the grouping of abstract ideas mental processes, i.e., a concept performed in the human mind, evaluation, judgement, and/or opinion of the user.
Step 2A, Prong Two:
The 2019 PEG defines the phrase “integration into a practical application” to require an additional step or a combination of additional steps in the claim to apply, rely on, or use the judicial exception. In addition, with respect to the system and computer-readable medium claims of claims 12-20, the mere recitation of a generic processor, memory, or storage medium to perform/store programming instructions of the recited/identified abstract idea does not integrate the identified abstract idea into a practical application. Accordingly, the above-mentioned additional elements/limitations do not integrate the abstract idea into a practical application; and therefore, the independent claims recite an abstract idea.
Step 2B:
Because the claims fail under Step 2A, the claims are further evaluated under Step 2B. The claims herein do not include additional elements that are sufficient to amount to significantly more than the judicial exception, because as discussed above with respect to integration of the abstract idea into practical application, the additional elements/limitations to perform the recited steps, amount to no more than insignificant extra-solution activity. Mere instructions to apply an exception using a generic component cannot provide an inventive concept. Therefore, independent claims 1 and 12 are not patent eligible. In addition, claims 2-11 and 13-20 of the instant application provide limitations that both individually or in combination do not integrate the identified abstract idea into a practical application or provide significantly more than the identified abstract idea.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 7-9, 12 and 18-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Jeong et al. (US 11,627,370 B1).
Regarding claim 1, Jeong teaches a region of interest extraction method comprising:
confirming whether a region of interest of a current frame is a cluster-centered region of interest or an object-centered region of interest (Col. 12, lines 29-37, user device may perform operation for detecting a target object in each image frame of video contents and defining a region of interest (ROI) based on the detected target object. In case of multiple target objects detected, the ROI may be defined to include all the target objects. Col. 24, lines 45-54, user device may determine whether single target object is detected or multiple target objects are detected);
extracting a tracking-based region of interest of the current frame according to a type of the region of interest (Col. 24, lines 55-59, when a single object is detected, user device may obtain a center coordinate of the target object and define a crop region to include the detected object (i.e., extract a tracking-based region of interest). Col. 25, lines 13-18, when multiple target objects are detected, user device may obtain center coordinates of the multiple target objects and define a crop region to include all the target objects (i.e., extract a tracking-based region of interest)); and
determining a final region of interest of the current frame based on the extracted tracking-based region of interest of the current frame (Col. 25, lines 26-34, user device may determine whether the defined crop region includes the ROI. When the defined crop region includes the ROI, user device displays the crop region as the second video (i.e., final region of interest)),
wherein, in response to the type of the region of interest being confirmed as the cluster-centered region of interest, extracting the tracking-based region of interest is performed for each cluster comprised of multiple objects in the current frame (Col. 25, lines 13-18, when multiple target objects are detected, user device may obtain center coordinates of the multiple target objects and define a crop region to include all the target objects (i.e., extract a tracking-based region of interest comprised of multiple objects)), and
wherein, in response to the type of the region of interest being confirmed as the object-centered region of interest, extracting the tracking-based region of interest is performed based on a single object (Col. 24, lines 55-59, when a single object is detected, user device may obtain a center coordinate of the target object and define a crop region to include the detected object (i.e., extract a tracking-based region of interest)).
Regarding claim 7, Jeong teaches the method of claim 1,
wherein the single object is determined differently depending on a type of the current frame (Jeong, Col. 13, lines 48-64; Fig. 11, detecting a target object in a first image frame (i.e., face of a female actor), error occurring in detecting the target object in a second frame (i.e., target object is female actor’s face but the face of male actor is detected)).
Regarding claim 8, Jeong teaches the method of claim 7,
wherein, in response to the current frame being an initialization frame, the single object is determined based on a user-specified signal or a result of multi-object tracking (Jeong, Col. 13, lines 48-64; Fig. 11, detecting a target object in a first image frame (i.e., face of a female actor)), and
wherein, in response to the current frame being a frame after the initialization frame, the single object is determined to be the single object determined in the initialization frame (Jeong, Col. 13, lines 48-64; Fig. 11, detecting a target object in a first image frame (i.e., face of a female actor), error occurring in detecting the target object in a second frame (i.e., target object is female actor’s face but the face of male actor is detected), correcting the error in a third image frame (i.e., detecting the female actor’s face)).
Regarding claim 9, Jeong teaches the method of claim 8,
wherein the initialization frame is a first frame of a frame sequence including the current frame or a first frame input after the type of the region of interest is changed (Jeong, Col. 22, lines 3-14, when the scene is changed, user device may perform an initialization process with the first image frame of each scene).
Claims 12, 18 and 19 recite systems with elements corresponding to the steps recited in Claims 1, 7 and 8, respectively. Therefore, the recited elements of these claims are mapped to the proposed reference in the same manner as the corresponding steps in their corresponding method claims.
Claim 20 recites a computer-readable storage medium storing a program with instructions corresponding to the steps recited in Claim 1. Therefore, the recited programming instructions of this claim are mapped to the proposed reference in the same manner as the corresponding steps in its corresponding method claim. Additionally, the Jeong reference discloses a computer readable storage medium (Col. 3, lines 53-55, a non-transitory computer readable medium may be provided for storing computer readable instructions).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 2 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Jeong et al. (US 11,627,370 B1) in view of Grancharov et al. (US 2021/0264619 A1).
Regarding claim 2, Jeong teaches the method of claim 1, as described above.
Although Jeong teaches calculating a travel path vector of a secondary object closely related to the target object in a previous image frame to the same secondary object in a current frame and compensate the coordinate of the target object in the current image based on the calculated travel path vector (Jeong, Col. 8, lines 3-17), Jeong does not explicitly teach “wherein a center point of the final region of interest of the current frame is determined based on movement paths of center points of final regions of interest of previous frames of the current frame” and “wherein a width and height of the final region of interest of the current frame are determined based on widths and heights of the final regions of interest of the previous frames of the current frame”. However, in an analogous field of endeavor, Grancharov teaches a typical implementation of the object tracking would be to start from the same coordinate and size of the bounding box in a frame (i.e., current frame) as the bounding box in a previous frame, preferably the most previous frame (Grancharov, Para. [0086]). The coordinated could, for example, represent the center of the bounding box. The size of the bounding box could be defined by a width and a height of the bounding box (Grancharov, Para. [0081]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Jeong with the teachings of Grancharov by including determining the center point, width and height of the region of interest (i.e., bounding box) of the current frame based on the center point, width and height of the region of interest (i.e., bounding box) in a previous frame. One having ordinary skill in the art before the effective filing date would have been motivated to combine these references because doing so would allow for efficient object tracking implementation in real-time applications, as recognized by Grancharov. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Claim 13 recites a system with elements corresponding to the steps recited in Claim 2. Therefore, the recited elements of this claim are mapped to the proposed combination in the same manner as the corresponding steps in its corresponding method claim. Additionally, the rationale and motivation to combine the Jeong and Grancharov references, presented in rejection of Claim 2, apply to this claim.
Claims 3 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Jeong et al. (US 11,627,370 B1) in view of Watanabe et al. (US 2019/0017811 A1).
Regarding claim 3, Jeong teaches the method of claim 1, as described above.
Although Jeong teaches obtaining a center coordinate of the target object in each image frame (Jeong, Col. 13, lines 1-3), Jeong does not explicitly teach “wherein a center point of the tracking-based region of interest extracted for each cluster is determined by an average position or median value of multiple objects in a cluster”. However, in an analogous field of endeavor, Watanabe teaches the center point may be calculated, for example, as a weighted average of the coordinates of a plurality of tracking points CP arranged in the region of interest (i.e., average of multiple objects in a cluster) (Watanabe, Para. [0075]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Jeong with the teachings of Watanabe by including calculating the center of the region of interest by determining the average of the coordinates of the multiple objects (i.e., plurality of tracking points) in the cluster. One having ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to combine these references because doing so would allow for analyzing regions of interest in moving objects, as recognized by Watanabe. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Claim 14 recites a system with elements corresponding to the steps recited in Claim 3. Therefore, the recited elements of this claim are mapped to the proposed combination in the same manner as the corresponding steps in its corresponding method claim. Additionally, the rationale and motivation to combine the Jeong and Watanabe references, presented in rejection of Claim 3, apply to this claim.
Claims 4-5 and 15-16 are rejected under 35 U.S.C. 103 as being unpatentable over Jeong et al. (US 11,627,370 B1) in view of Watanabe et al. (US 2019/0017811 A1), as applied to claims 3 and 14 above, and further in view of Loer et al. (US 10,460,495 B1).
Regarding claim 4, Jeong in view of Watanabe teaches the method of claim 3, as described above.
Although Jeong in view of Watanabe teaches determining a center point based on a weighted average of tracking point locations (Watanabe, Para. [0075]), they do not explicitly teach “wherein a width and height of the tracking-based region of interest extracted for each cluster are determined based on the center point and a radius of the cluster”. However, in an analogous field of endeavor, Loer teaches each disjoint region is a square with a side length that is within a threshold number of pixels of twice the radius of the bounding geometry circles (i.e., clusters). For example, if the bounding geometry circles have an average radius of 16 pixels, the disjoint regions may each be 30 pixels wide and 30 pixels long (Loer, Col. 15, lines 1-13).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Jeong in view of Watanabe with the teachings of Loer by including that the width and height of the tracking-based region of interest (i.e., disjoint region) is determined based on center point and radius of the cluster (i.e., radius of the bounding geometry circles). One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for real-time tracking of objects, as recognized by Loer. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Regarding claim 5, Jeong in view of Grancharov further in view of Watanabe teaches the method of claim 4,
wherein the determined width and height are adjusted to include all bounding boxes of the multiple objects in the cluster (Jeong, Col. 25, lines 13-18, when multiple target objects are detected, user device may obtain center coordinates of the multiple target objects and define a crop region to include all the target objects).
Claims 15 and 16 recite systems with elements corresponding to the steps recited in Claims 4 and 5, respectively. Therefore, the recited elements of these claims are mapped to the proposed combination in the same manner as the corresponding steps in their corresponding method claims. Additionally, the rationale and motivation to combine the Jeong, Watanabe and Loer references, presented in rejection of Claim 4, apply to these claims.
Claims 6 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Jeong et al. (US 11,627,370 B1) in view of Watanabe et al. (US 2019/0017811 A1) further in view of Loer et al. (US 10,460,495 B1), as applied to claims 4-5 and 15-16 above, and further in view of Baldacci et al. (US 2019/0122040 A1).
Regarding claim 6, Jeong in view of Watanabe further in view of Loer teaches the method of claim 5, as described above.
Although Jeong in view of Watanabe further in view of Loer teaches the size of the region of interest is based on the radius of the cluster (Loer, Col. 15, lines 1-13), they do not explicitly teach “wherein the radius of the cluster is set in consideration of an output resolution of an image including the current frame”. However, in an analogous field of endeavor, Baldacci teaches a cluster estimator that calculates an expected target pixel cluster size (i.e., sets radius of the cluster) using a priori information such as camera geometry, camera location, image resolution (output resolution of an image including the current frame), field of view of the camera, and distance of the camera from the target (Baldacci, Para. [0120]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Jeong in view of Grancharov further in view of Loer with the teaches of Baldacci by including determining the radius of the cluster in consideration of the output image resolution. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for efficiently analyzing frames of image data, as recognized by Baldacci. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Claim 17 recites a system with elements corresponding to the steps recited in Claim 6. Therefore, the recited elements of this claim are mapped to the proposed combination in the same manner as the corresponding steps in its corresponding method claim. Additionally, the rationale and motivation to combine the Jeong, Watanabe, Loer and Baldacci references, presented in rejection of Claim 6, apply to this claim.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Jeong et al. (US 11,627,370 B1) in view of Kazuki Takemoto (US 2020/0098149 A1).
Regarding claim 10, Jeong teaches the method of claim 9, as described above.
Although Jeong teaches detecting a target object in an image frame (Jeong, Col. 13, lines 48-64), Jeong does not explicitly teach “wherein the user-specified signal includes a signal that specifies a boundary of the single object through a mouse drag function”. However, in an analogous field of endeavor, Takemoto teaches the user is able to designate a region of the object, by carrying out a drag operation of the mouse (Takemoto, Para. [0040]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Jeong with the teachings of Takemoto by including specifying the boundary of the object through a mouse drag operation. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for a user to select a target object region, as recognized by Takemoto. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Jeong et al. (US 11,627,370 B1) in view of Kazuki Takemoto (US 2020/0098149 A1), as applied to claim 10 above, and further in view of Baldacci et al. (US 2019/0122040 A1).
Regarding claim 11, Jeong in view of Takemoto teaches the method of claim 10, as described above.
Although Jeong in view of Takemoto teaches detecting a target object in an image frame (Jeong, Col. 13, lines 48-64), they do not explicitly teach “wherein a width and height of the tracking-based region of interest extracted based on the single object are set in consideration of an output resolution of an image including the current frame”. However, in an analogous field of endeavor, Baldacci teaches a cluster estimator that calculates an expected target pixel cluster size (i.e., sets radius of the cluster) using a priori information such as camera geometry, camera location, image resolution (output resolution of an image including the current frame), field of view of the camera, and distance of the camera from the target (Baldacci, Para. [0120]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Jeong in view of Takemoto with the teaches of Baldacci by including determining the radius of the cluster in consideration of the output image resolution. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for efficiently analyzing frames of image data, as recognized by Baldacci. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Emma Rose Goebel whose telephone number is (703)756-5582. The examiner can normally be reached Monday - Friday 7:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached at (571) 272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Emma Rose Goebel/Examiner, Art Unit 2662
/AMANDEEP SAINI/Supervisory Patent Examiner, Art Unit 2662