Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed December 29, 2025 have been fully considered but they are not persuasive.
Regarding claim 1,
(1) applicant states that “Inventor merely discloses "executing the user behaviour analysis method of financial institution security system". In other words, the 'behaviour analysis network' (a neural network) in Inventor processes specific physical inputs: step length, step frequency, human body posture data, and expression information. Thus, Inventor focuses on the kinematics and facial expression of the user in isolation to predict a transaction. Crucially, Inventor does not teach inputting an “identified relationship” (e.g., a semantic link defined by graph data between a person and a separate object or person) into its machine learning model. Inventor's model inputs are intrinsic to the user (posture/face), not extrinsic relationships with the environment.”. Examiner disagrees with this statement. Inventor proposes to input not just the kinematics and facial expression of the user in isolation into the machine learning model, but also an identified, extrinsic relationship. Inventor proposes to receive a video of the user entering the preset area (receiving a section of video entering the preset area by the user. Abstract) and the information regarding the preset area, e.g. the bank hall, is input into the machine learning model to predict a future behavior of the user (Wherein, the preset area can be the bank hall, the hall of the government department, the subway, the railway station and so on. pre-accurately predicting the behaviour of the user (item needed to be processed), on the one hand, improving the service effect and service efficiency of the service of the bank service, municipal service and so on, improving the bank operation, validity and efficiency of the municipal work; on the other hand, it also can timely according to the predicted action (item to be processed), preventing the occurrence of security problem, improving security effect of hall, government department hall, subway, railway station and so on, ensuring the safety of the user. Page 7 2nd paragraph). In the broadest sense, a bank hall is an object. Therefore, a person at a bank hall defines an identified, extrinsic relationship between the person and the object.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 5-12 and 14-17 are rejected under 35 U.S.C. 103 as being unpatentable over Wnuk (US Patent Pub. No. US 2015/0363644 A1), in view of Inventor (Chinese Patent Publication No.: CN 111476202 A), hereinafter Inventor.
Regarding claim 1, Wnuk teaches a non-transitory computer-readable recording medium having stored therein an information processing program that causes a computer to execute a process, the process comprising: acquiring video data that contains a plurality of frames as a single processing unit (Although a single sensor 120 is shown ( e.g., a video camera sensor), sensor 120 may represent one or more sensors capable of converting observations of an activity into digital representation 130, possibly according to multiple data modalities (i.e., according to multiple modes of existence, experience or expression). [0051]) and that includes target objects including a person and an object (In the example shown, an ice skater is illustrated as moving around an environment. [0053]); first identifying each relationship between the target objects (In the example shown, an ice skater is illustrated as moving around an environment. [0053]) from the plurality of frames contained in the acquired video data (Digital representation 130 comprises a video data stream with a number of frames, wherein the ice skater may be located in a different location from frame-to-frame. [0053]), by using graph data (An activity graph can be constructed to indicate causality of temporal nodes 141 ( clusters of temporal features 135). [0057]) that defines a relationship between a person and another person or a relationship between a person and an object (In the example shown, an ice skater is illustrated as moving around an environment. Digital representation 130 comprises a video data stream with a number of frames, wherein the ice skater may be located in a different location from frame-to-frame. [0053]); second identifying each of the elemental behaviors in the video data (In the example shown, an ice skater is illustrated as moving around an environment. Digital representation 130 comprises a video data stream with a number of frames, wherein the ice skater may be located in a different location from frame-to-frame. The ice skater can be considered to be tracing out a volume in an (X, Y, T) space, where T corresponds to time, where X and Y represent the 2D space of the image frames. [0053]) by using higher-level behavior identification rule (Based on similarity activity scores 250, activity recognition device 210 may access activity recognition result set 260. [0086]. Also see Figure 4.
PNG
media_image1.png
950
1028
media_image1.png
Greyscale
) that records each of the elemental behaviors in association with a basic action (The disclosed approach provides infrastructure for a computing device to recognize one or more activities represented in a digital representation of a scene. The activities could include one or more activities across a broad spectrum of action. Example activities include plays, sports, shopping, game play, military training, physical therapy, or other types of behaviors. [0049]. Activity 110 is ingested by using one or more feature detection algorithms to generate a plurality of features
133 from digital representation 130. [0054]) and the current behavior in association with a change of elemental behaviors (The temporal features 135 are converted into one or more activity graphs 140 comprising nodes 141 that represent clusters of temporal features 135. Activity graphs 140 can describe temporal or spatial relationships among comparable events in time (e.g., a motion, a sound, etc.). [0055]), which are performed to identify the current behavior (Activity 110 is ingested by using one or more feature detection algorithms to generate a plurality of features 133 from digital representation 130. [0054]. Thus, ingestion metadata 145 could comprise domain-specific attributes ( e.g., attributes related to a medical domain, health care domain, sports domain, gaming domain, shopping domain, etc.), object attributes ( e.g., type of object, name of object, etc.), environment or context attributes ( e.g., location attributes, time attributes, position attributes, orientation attributes, etc.) or other types of attributes. [0064]); and predicting the identified future behavior of the person (In such cases, activity recognition device 210 could be configured to predict a next action within an observed activity based on the similarity activity scores 250. The scores could indicate a probability of the next action matching a next action within the known activity graphs. Thus, activity recognition result set 260 could include a prediction with respect to observed activity graph 240. [0088]).
Wnuk does not teach the following limitations as further recited, but Inventor further teaches identify the current behavior (receiving a section of video entering the preset area by the user; according to the image in the first section of video, obtaining the human body posture information of the user in each image and expression information of the user. Abstract), by using skeleton information (Optionally, the human body posture information comprises a human body skeleton map. Page 5 2nd paragraph), indicating joint position of each person (calculating the score of each human key point, taking the score of human key point as human gesture data. Page 8 6th paragraph), identified from the plurality of frames, respectively (receiving a section of video entering the preset area by the user. Abstract); and predicting the identified future behavior of the person (Therefore, the accuracy of the user behaviour is predicted according to the gesture information and expression information of the user. Abstract) including an emotion (obtaining the expression information of the user based on the face topological graph. Page 5 6th paragraph) and a state affected by an internal state of the person (it is according to the gesture information and expression information of the user last step predicting the next step of behavior situation, predicting the user wants to do what (which reads on “a state affected by an internal state of the person”.). Abstract), by inputting the identified behavior of the person (it is according to the gesture information and expression information of the user last step predicting the next step of behavior situation. Abstract) and at least one of the identified relationship between a person and another person (Note: the claim language is interpreted as disjunctive based on the specification (identifies a relationship between a person and another person or a relationship between a person and an object [0018]).) and the identified relationship between a person and an object (Wherein, the preset area can be the bank hall, the hall of the government department, the subway, the railway station and so on. pre-accurately predicting the behaviour of the user (item needed to be processed), on the one hand, improving the service effect and service efficiency of the service of the bank service, municipal service and so on, improving the bank operation, validity and efficiency of the municipal work; on the other hand, it also can timely according to the predicted action (item to be processed), preventing the occurrence of security problem, improving security effect of hall, government department hall, subway, railway station and so on, ensuring the safety of the user. Page 7 2nd paragraph. In the broadest sense, a bank hall is an object. Therefore, a person at a bank hall defines an identified, extrinsic relationship between the person and the object.) to a machine learning model (The user behaviour analysis method of financial institution security system is applied to the artificial intelligence field, namely executing the user behaviour analysis method of financial institution security system through the robot, so as to advance a corresponding service for the user. Page 12 last paragraph. FIG. 2 is a structure schematic diagram of a behaviour analysis network provided by the embodiment of the present invention.
PNG
media_image2.png
947
1354
media_image2.png
Greyscale
).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Wnuk to incorporate the teachings of Inventor to identify the current behavior, by using skeleton information, and predicting the identified future behavior of the person including an emotion and a state affected by an internal state of the person, by inputting the identified behavior of the person and at least one of the identified relationship between a person and another person and the identified relationship between a person and an object to a machine learning model in order to increase the behavior analysis prediction accuracy.
Claims 2-3 and 5-8, unamended and are rejected based on the combination of Wnuk, in view of Inventor. The grounds of rejection established in the last Office Action is fully incorporated herein.
Method claim 9 is drawn to the method of using the non-transitory computer-readable recording medium having stored therein an information processing program that causes a computer to execute a process as claimed in claim 1. Therefore method claim 9 corresponds to the non-transitory computer-readable recording medium process claim 1 and is rejected for the same reasons of obviousness as used above.
Apparatus claims 10-12 and 14-17 are drawn to the apparatus corresponding to the non-transitory computer-readable recording medium having stored therein an information processing program that causes a computer to execute a process as claimed in claims 1-3 and 5-8. Therefore apparatus claims 10-12 and 14-17 correspond to the non-transitory computer-readable recording medium process claims 1-3 and 5-8, and are rejected for the same reasons of obviousness as used above.
Claims 4 and 13, unamended and are rejected based on the combination of Wnuk (US Patent Pub. No. US 2015/0363644 A1), in view of Inventor (Chinese Patent Publication No.: CN 111476202 A), hereinafter Inventor, further in view of Filntisis (Fusing Body Posture With Facial Expressions for Joint Recognition of Affect in Child–Robot Interaction, IEEE ROBOTICS AND AUTOMATION LETTERS, VOL. 4, NO. 4, OCTOBER 2019), hereinafter Filntisis, further in view of Psaltis (Multimodal Affective State Recognition in Serious Games Applications, Proc. IEEE Int. Conf. Imag. Syst. Techn., 2016, pp. 435–439), hereinafter Psaltis. The ground of rejection established in the last Office Action is fully incorporated herein.
Apparatus claim 13 is drawn to the apparatus corresponding to the non-transitory computer-readable recording medium having stored therein an information processing program that causes a computer to execute a process as claimed in claim 4. Therefore apparatus claim 13 corresponds to the non-transitory computer-readable recording medium process claim 4, and is rejected for the same reasons of obviousness as used above.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LEI ZHAO whose telephone number is (703)756-1922. The examiner can normally be reached Monday - Friday 8:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, VU LE can be reached at (571)272-7332. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LEI ZHAO/Examiner, Art Unit 2668
/VU LE/Supervisory Patent Examiner, Art Unit 2668