DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
The examiner acknowledges receipts of remarks dated April 7, 2026.
Response to Arguments
Applicant's arguments filed April 7, 2026 have been fully considered but they are not persuasive.
In regards to 1, the applicants state that “At no point does Nakamura identify a relationship between an object, such as a product or a tool, and a person, let alone use such a relationship to determine abnormal behavior.” The examiner respectfully traverses. In response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., product or a tool) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). In particular, the prior art teaches determining relationship between a child and a parent in which, the child is interpreted as the object of interest. Therefore, prior art does indeed teach determining a relationship between the object (a child) and a person (the parent).
Furthermore, the applicants state that the applied art does not teach or suggest “"determining whether the person has performed an abnormal behavior on a product." Nakamura's system determines whether a child has become separated from a parent or guardian, not whether a person has performed an abnormal behavior on a product.” The examiner respectfully disagrees. The prior art teaches an abnormal behavior in the adult/child relationship, of the adult losing or separating from the child. As a parent or adult losing or separating from a child would amount to abnormal behavior. Therefore, the examiner submits that prior art Nakamura teaches all the limitations of claim 1.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1, 2, 3, 5, 7, 9 and 12-14 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by JP2018201176A by Nakamura et al. (hereinafter ‘Nakamura’). Translation of the Japanese application used for this rejection is included with office action.
In regards to claim 1, Nakamura teaches a non-transitory computer-readable recording medium having stored therein an information processing program that causes a computer to execute a process, the process comprising: acquiring a video that is captured by one or more camera apparatuses; (See Nakamura page 2, Nakamura teaches capturing videos using cameras.)
identifying a relationship for identifying a behavior between an object and a person included in the video by analyzing the acquired video; (See Nakamura page 4, Nakamura teaches determining relationships between persons in identified in the videos)
determining whether the person has performed an abnormal behavior on a product on an outside of an imaging range of the camera apparatus based on the identified relationship; and giving an alert based on a determination result on whether the person has performed the abnormal behavior on the product on the outside of the imaging range. (See Nakamura page 5, Nakamura teaches determining abnormal behavior such as separation from parent and outputting an alert.)
In regards to claim 2, Nakamura teaches the process further including: acquiring a plurality of videos that are captured by a plurality of camera apparatuses installed in a store and that include different areas captured by the plurality of camera apparatuses; identifying a first relationship for identifying a correlation between an object and the person included in a video in which a first area is captured by analyzing the video in which the first area is captured among the plurality of acquired videos; identifying a second relationship for identifying a correlation between an object and the person included in a video in which a second area is captured by analyzing the video in which the second area is captured among the plurality of acquired videos; (See Nakamura page 4, Nakamura teaches determining second relationship using second cameras.)
determining whether the person has performed an abnormal behavior on a product in an area that is located between the first area and the second area and that is located on an outside of imaging ranges of the plurality of camera apparatuses based on the first relationship and the second relationship; and giving an alert if it is determined that the person has performed the abnormal behavior. (See Nakamura page 5, Nakamura teaches determining abnormal behavior such as separation from parent and outputting an alert.)
In regards to claim 3, Nakamura teaches wherein a time of the video in which the second area is captured is later than a time of the video in which the first area is captured. (See Nakamura page 3).
In regards to claim 5, Nakamura teaches the process further including: identifying a first person for whom the identified relationship temporally changes from a first relationship to a second relationship based on the acquired video, wherein the determining whether the person has performed an abnormal behavior on the product includes determining whether the first person has performed an abnormal behavior on the product on the outside of the imaging range based on the identified relationship. (See Nakamura page 8)
In regards to claim 7, Nakamura the process further including: identifying an area that is an area in which the person has performed an abnormal behavior on the product, that is located between the first area and the second area, and that is located on an outside of imaging ranges of the plurality of camera apparatuses, based on the plurality of the camera apparatuses that have performed image capturing, wherein the giving the alert includes giving the alert indicating occurrence of abnormality on the product in association with the identified area that is located on the outside of the imaging ranges of the plurality of camera apparatuses. (See Nakamura page 5).
In regards to claim 9, Nakamura teaches wherein the identifying the first person includes generating a scene graph that identifies the relationship for each of the persons included in the video by inputting the acquired video to a machine learning model; and identifying the first person by analyzing the scene graph. (See Nakamura page 4).
In regards to claim 12, Nakamura teaches the process further including: identifying a position of the person included in each of the videos that are captured by the respective camera apparatuses 100 by a first index that is different for each of the camera apparatuses; identifying the positions of the persons identified by the first indices by using a second index that is common among the plurality of camera apparatuses; and determining whether the persons included in the respective videos are an identical person based on the positions of the persons identified by using the second index. (See Nakamura page 4).
Claims 13 and 14 recite limitations that are similar to that of claim 1. Therefore, claims 13 and 14 are rejected similarly as claim 1.
Allowable Subject Matter
Claims 4, 6, 8 and 10-11 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
In regards to claims 4, 6, 8 and 10-11, the applied art does not teach or suggest the claimed limitations:
In regards to claim 4, Nakamura does not teach wherein the first relationship indicates that the person holds the product and a predetermined object that is used for shoplifting of the product, and the second relationship indicates that the person holds the predetermined object that is used for shoplifting of the product, the process further including: determining, when the predetermined object that is held by the person in the first relationship is also held in the second relationship and when the product that is held in the person in the first relationship is not held in the second relationship, that the person has performed an abnormal behavior on the product in an area that is located between the first area and the second area and that is located on an outside of imaging ranges of the plurality of camera apparatuses.
In regards to claim 6, Nakamura does not teach wherein the identifying the relationship includes identifying, from the video, a first area including the object, a second area including the person, and a first relationship for identifying a correlation between the object included in the first area and the person included in the second area by inputting the acquired video to a machine learning model; and identifying, from the video, a third area including the object, a fourth area including the person, and a second relationship for identifying a correlation between the object included in the third area and the person included in the fourth area by inputting the acquired video to a machine learning model, and the determining whether the person has performed an abnormal behavior on the product includes, when the person included in the second area and the person included in the fourth area are identical, determining whether the person has performed an abnormal behavior on the product by comparing the identified first relationship, the identified second relationship, and a rule that is set in advance.
In regards to claim 8, Nakamura does not teach wherein the determining whether the person has performed an abnormal behavior on the product includes determining whether the person has performed an abnormal behavior including one of shoplifting and a behavior that leads to shoplifting on the product on an outside of an imaging range of the camera apparatus based on the identified first relationship and the identified second relationship.
In regards to claim 10, Nakamura does not teach wherein the identifying the relationship includes extracting a first feature value that corresponds to one of the object and the person from the video; detecting the object and the person included in the video from the extracted first feature value; generating a second feature value that is a combination of the plurality of detected objects, the plurality of detected persons, and the first feature value of one of the object and the person in at least a single pair of the object and the person; generating a first map that indicates the plurality of objects, the plurality of persons, and the relationship for identifying at least a single correlation between the object and the person based on the first feature value and the second feature value; extracting a fourth feature value based on a third feature value that is obtained by converting the first feature value and based on the first map; and identifying the relationship from the fourth feature value.
In regards to claim 11, Nakamura does not teach wherein the identifying the relationship includes generating skeleton information on the person by analyzing the acquired video; identifying the first relationship based on the generated skeleton information; and identifying the second relationship based on the generated skeleton information.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to UTPAL D SHAH whose telephone number is (571)272-5729. The examiner can normally be reached M-F: 7:30-5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vu Le can be reached at (571) 272-7332. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/UTPAL D SHAH/Primary Examiner, Art Unit 2668
Prosecution Insights