DETAILED ACTION
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . The following NON-FINAL office action is in response to Applicant communication filed on 12/22/2025 regarding application 17/979,773. Claims 1, 4, 6-9, 13-15 and 19 have been amended. Claim 2 has been canceled. Claims 20-24 have been added as new claims. Thus, Claims 1, 4-9, 11-15 and 17-24 have been rejected.
Response to Amendments
2. Applicant’s amendment filed on 12/22/2025 necessitated new grounds of rejection in this office action.
Foreign Priority
3. The Examiner has noted the Applicants claiming Priority from Foreign Application JP2022-025801 filed on 02/22/2022. Receipt is acknowledged of papers submitted under 35 U.S.C. § 119(a)-(d), which papers have been placed of record in the file. Therefore, Examiner notes the effective filing date of this application examined on the record is 02/22/2022.
Continued Examination under 37 CFR 1.114
4. A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 12/22/2025 has been entered.
Response to Arguments
5. Applicant’s arguments, see page 13 filed on 12/22/2025, with respect to the 35 U.S.C. § 112 (b) Claim Rejections for Claims 1-2, 4-9, 11-15 and 17-19 have been fully considered and are found to be persuasive. Therefore, the 35 U.S.C. § 112 (b) Claim Rejections for Claims 1-2, 4-9, 11-15 and 17-19 have been withdrawn.
6. Due to Applicant’s newly proposed amendments, Examiner adds 35 U.S.C. § 112 (d) rejections for Claims 9 and 15.
7. Applicant’s arguments, see pages 15-16 filed on 12/22/2025, with respect to the 35 U.S.C. § 103 Claim Rejections for Claims 1-2, 4-9, 11-15 and 17-19 have been fully considered and are found to be not persuasive. Applicant’s arguments with respect to Claims 1, 4-9, 11-15 and 17-24 have been considered, but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Response to 35 U.S.C. § 101 Arguments
8. Applicant’s 35 U.S.C. § 101 arguments, filed with respect to Claims 1, 4-9, 11-15 and 17-24 have been fully considered, but they are found not persuasive (see Applicant Remarks, Pages 13-15, dated 12/22/2025). Examiner respectfully disagrees.
Argument #1:
(A). Applicant argues that Claims 1, 4-9, 11-15 and 17-24 recite additional elements that integrate the judicial exception into a practical application under revised step 2a prong two of the 35 U.S.C. § 101 analysis (see Applicant Remarks, Pages 13-15, dated 12/22/2025). Examiner respectfully disagrees.
Specifically, Applicant argues that the amended claims are eligible under Step 2A Prong Two because they recite a specific improvement to the functioning of a computer in the context of tracking and grouping entities in the video data (see Applicant Remarks, 1st ¶ on Page 14, dated 12/22/2025). Examiner respectfully disagrees.
In response to Applicant remarks here for step 2a prong 2, Examiner notes that when factoring the additional elements both individually and as an ordered combination in view of the claim limitation steps for Independent Claims 1, 8 and 14 are patent ineligible because they represent mere instructions to apply an abstract idea using computer components, which does not amount to a practical application. Examiner provides a detailed breakdown of each of these steps. First, the step of “identifying relationships... by analyzing a video”: This is categorized as insignificant extra-solution activity. Video analysis is treated as a generic data collection step (input) that merely provides the "raw material" for the abstract idea. It does not improve video processing technology itself. Secondly, the claim limitation step of “identifying a plurality of customers who received customer services...”: This step is a mental process that has been automated. It lacks any technical detail on how the computer performs this identification differently or better than a human observer, making it a high-level functional goal rather than a technological solution. Moreover, the step of “classifying each of the customers... calculating a group similarity... adding a penalty value”: While this uses a mathematical concept (calculating similarity and adding a penalty), it does not improve the functioning of the computer. Examiner points out that applying math to a business problem (organizing customers) without a specific technical improvement to the algorithm's efficiency or the computer's architecture is considered a "mere instruction to apply the exception".
Fourth, the step of “identifying behaviors... and behavior types that define transition of processes... until purchase”: Tracking a "purchase process" is a fundamental business practice; automating it with a computer does not transform it into a practical application. “Generating information on the purchase... associating the certain group with behaviors”: These are generic computer functions (generating and storing data associations). They do not resolve a technical problem but rather provide the final "output" of the abstract business analysis. “Inputting a first partial image... into a machine learning model... as a feature value... into a neural network”: This is the most "technical" limitation, yet it remains ineligible because it recites generic machine learning. Simply using a neural network as a tool to perform identification (a task humans can do) without claiming a specific, novel architecture or training method that improves the ML field itself is insufficient for integration. Lastly, “determining whether the first person is one of the sales clerk and the plurality of customers”: This is the final result or "judgment" of the process. Under the "Apply It" consideration, a claim that merely recites an abstract idea and says "determine the result" does not impose a meaningful limit on the exception.
In summary, the steps for Independent Claims 1, 8 and 14 are ineligible because, taken as a whole, they describe a commercial/mental process and use generic computer and AI components as "tools" to perform that process. Because no step provides a "specific, technical solution to a technical problem" (such as a faster video processing algorithm or a more efficient neural network architecture), the abstract idea is not integrated into a practical application under step 2a prong 2 of the 35 U.S.C. § 101 analysis. Examiner maintains that Claims 1, 4-9, 11-15 and 17-24 as currently recited do not contain additional elements that integrate the judicial exception into a practical application under step 2a prong 2 of the 35 U.S.C. 101 analysis.
Argument #2:
(B). Applicant argues that Claims 1, 4-9, 11-15 and 17-24 are analogous to Ex parte Desjardins (Appeal 2024-000567) stating that a precedential decision holding that claims improving an AI model’s performance are patent eligible. Similarly, Claim 1 recites a specific technical solution to a computer vision problem: the problem of falsely grouping unrelated individual who are in close proximity (e.g., at a service counter) (see Applicant Remarks, 2nd ¶ of Page 14, dated 12/22/2025). Examiner respectfully disagrees.
The rebuttal argument for patent ineligibility under 35 U.S.C. § 101 Step 2A, Prong 2 would assert that the claim steps are non-analogous to eligible cases like Ex parte Desjardins. The precedential decision Ex parte Guillaume Desjardins (PTAB Sept. 26, 2025; designated precedential Nov. 4, 2025) represents a shift toward broader eligibility for AI inventions that improve the technical functioning of a model. However, the provided steps for customer behavior analysis remain ineligible under 35 U.S.C. 101 Step 2A, Prong 2 because they are fundamentally non-analogous to the technological improvements found in Desjardins.
The core of the rebuttal is that the current claims apply generic machine learning to a business problem, whereas Desjardins claimed a technical solution to a problem within machine learning itself. Desjardins Holding: The invention in Desjardins addressed the technical problem of "catastrophic forgetting" in neural networks. The claims recited a specific training method (using importance measures and penalty terms) that improved how the model itself operates by preserving knowledge across tasks. The Current Claim: The steps of "identifying relationships," "classifying customers," and "generating purchase information" describe a method of organizing human activity (business analytics). Unlike Desjardins, these steps do not improve the model's efficiency, storage, or learning architecture; they merely use a model to process specific types of data. Reason #2: Generic Application vs. Specific Training Process. Desjardins Holding: The PTAB found eligibility because the claims were not just "using" AI, but were directed to a specific training process that enhanced model generalization and performance. The Current Claim: Inputting a "first partial image" into a "neural network" to determine if a person is a "sales clerk" is a generic classification task. It does not modify the underlying AI architecture or training methodology to solve a technical problem. Applying existing machine learning to a new data environment (like store surveillance) without improving the AI's internal operations is considered a "mere instruction to apply an abstract idea”. Reason #3: Business/Mental Process vs. Computer/AI Field Improvement. Desjardins Holding: The Desjardins panel emphasized that the improvement was in the technical field of machine learning systems, not a business domain. The Current Claim: The additional elements (calculating "group similarity" and adding a "penalty value" to group customers) are directed to commercial/marketing analytics. While they use a "penalty value" (terminologically similar to Desjardins), in this context, it is used to force a business outcome (separating clerks from customers) rather than to technically optimize the model's objective function for general knowledge preservation.
Examiner points out that in the Ex parte Guillaume Desjardins (PTAB Sept. 26, 2025; designated precedential Nov. 4, 2025) case the problem type was Technical (Catastrophic Forgetting), the claimed focus was “how the model learns” and the “Outcome – improved system efficiency/storage”. In contrast to the Ex parte Guillaume Desjardins (PTAB Sept. 26, 2025; designated precedential Nov. 4, 2025) case, the claim limitation steps of Independent Claims 1, 8 and 14 problem type pertained to Business (Customer Classification), the claimed focus was on “What data the model processes” and thirdly the outcome was “generated of purchase information”.
Because the current claims of the instant application focused on using a neural network to facilitate a marketing analysis rather than improving the technical operation of the machine learning system, they do not integrate the abstract idea into a practical application under the Desjardins framework and are non-analogous under the 35 U.S.C. § 101 analysis guidelines.
Argument #3:
(C). Applicant argues that Claims 1, 4-9, 11-15 and 17-24 are analogous to Example 47 Claim 3 of the USPTO Subject Matter Eligibility Examples whereby the claim recites a specific manner of data processing (manipulating similarity scores with a penalty) to achieve a practical application (accurate group tracking in a store) that cannot be performed in the human mind (see Applicant Remarks, last ¶ of Page 14, dated 12/22/2025). Examiner respectfully disagrees.
Examiner discloses that Example 47, Claim 3 of the USPTO Subject Matter Eligibility Examples integrates an abstract idea into a technological improvement (network security), whereas the current claim limitation steps of Independent Claims 1, 8 and 14 merely uses generic AI to automate a business/marketing process.
Moreover, in Example 47, Claim 3, this claim is eligible because it solves a technical problem in computer networking. It recites specific post-detection steps—dropping malicious packets and blocking traffic from source addresses in real time—which proactively remediate a network attack and improve the functioning of the network system itself. In contrast to Example 47, Claim 3 of the USPTO Subject Matter Eligibility Examples, the steps of "calculating group similarity," "adding a penalty value," and "generating purchase information" do not improve the computer's functioning. Instead, they optimize a commercial interaction model (identifying which customers were served by which clerks). Under the "Apply It" consideration, using an ANN to facilitate a business outcome (sales analytics) is a generic application of technology, not a technological improvement to the field of video processing or AI. Reason #2: Absence of Remedial Technical Action. Example 47, Claim 3 eligibility hinges on the remedial actions that occur after anomaly detection (e.g., dropping packets). These actions directly interact with the physical/virtual network hardware to ensure its security and operational integrity. In contrast to Example 47 Claim 3 of the USPTO Subject Matter Eligibility Examples, the steps of "associating" and "generating information" are purely informational or record-keeping. There is no automated, proactive technological response recited (e.g., a technical change to the camera system or store network) that is analogous to the "dropping" or "blocking" actions that saved the network in Example 47. Reason #3: Nature of the "Penalty Value”. The current claim limitation steps of Independent Claims 1, 8 and 14 use of a "penalty value" to ensure customers and clerks belong to "different groups" which is a logical constraint on a business data set. In contrast, eligible AI claims (like those in Ex parte Desjardins or the principles behind Example 47) typically use mathematical penalties to solve internal model failures (like catastrophic forgetting or model convergence). Here, the penalty is simply a tool for a mental-like classification of human roles, making it an abstract method of organizing human activity.
In summary, the current steps are ineligible because they fail to provide the "meaningful limit" required for integration into a practical application. Unlike the network-security improvement in Example 47, Claim 3, these claims of the instant application describe a business-analytic result achieved through conventional video and AI processing, which remains "directed to" an abstract idea and are hereby non-analogous.
Argument #4:
(D). Applicant argues that Claims 1, 4-9, 11-15 and 17-24 recite additional elements that amount to significantly more than the recited judicial exception under step 2B of the 35 U.S.C. § 101 analysis (see Applicant Remarks, 1st ¶ of Page 15 dated 12/22/2025). Examiner respectfully disagrees.
In response, Examiner refers Appellant to Examiner’s 35 U.S.C. 101 analysis section shown below particularly for Independent Claims 1, 8 and 14. The claims do not recite additional elements that amount to significantly more than the recited judicial exceptions, because they are merely directed to the particulars of the abstract idea and likewise do not add significantly more to the above-identified judicial exceptions. The limitations are directed to limitations referenced in MPEP § 2106.05I.A. that are not enough to qualify as significantly more when recited in these claims with the abstract idea which include: (1) adding the words “apply it” (or an equivalent) with the judicial exception, (2) or mere instructions to implement an abstract idea on a computer and providing the results to the user on a computer, and (3) generally linking the use of the judicial exception to a particular technological environment or field of use.
Under the 2025 Federal Circuit decision in Recentive Analytics, Inc. v. Fox Corp., patents that merely apply generic machine learning to new data environments without disclosing technical improvements to the models themselves are patent-ineligible. Each step of the processes for Independent Claims 1, 8 and 14 fails Step 2B for the following reasons: Generic Data Collection and Processing: “analyzing a video” and “inputting a first partial image... into a machine learning model”: The Federal Circuit has established that merely collecting and analyzing data using generic computing infrastructure does not constitute an inventive concept. Automation of Mental/Human Tasks: “identifying relationships,” “identifying customers,” and “identifying behaviors”: These steps are functional analogs of what a human floor manager or security guard does mentally. Automating these human-performed observations with a computer does not provide "significantly more" than the abstract idea itself. “Calculating a group similarity... and adding a penalty value”: While these recite mathematical operations or mental processes, they are applied in a conventional manner to a classification problem. Without a specific, technical improvement to the neural network’s architecture—such as the training method for preserving knowledge found in Ex parte Desjardins—these are viewed as "mere instructions to apply the exception" using a known tool. Business-Focused Results: “generating information on the purchase” and “associating the certain group with behaviors”: These are generic output and storage functions. They solve a business problem (marketing/sales tracking) rather than a technical problem in computer hardware or software. “Determining whether the first person is a sales clerk”: This final judgment is the result of using the machine learning tool as intended. The use of an ANN for classification is an activity that does not transform an abstract idea into a patent-eligible invention.
The steps of the instant application are ineligible because, taken as an ordered combination, they do not improve the technology of machine learning or video analysis; they only use that technology to automate methods of organizing human activity. Because no element or combination of elements provides a "concrete, technical innovation" beyond standard AI application, the claim lacks an inventive concept and is ineligible under 35 U.S.C. § 101 step 2B.
Claim Rejections - 35 USC § 112
9. The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
The following is a quotation of pre-AIA 35 U.S.C. 112, fourth paragraph:
Subject to the following paragraph [i.e., the fifth paragraph of pre-AIA 35 U.S.C. 112], a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
10. Claims 9 and 15 are rejected under 35 U.S.C. 112(d) or pre-AIA 35 U.S.C. 112, 4th paragraph, as being of improper dependent form for failing to further limit the subject matter of the claim upon which it depends, or for failing to include all the limitations of the claim upon which it depends.
(A). Dependent Claims 9 and 15 recite the following claim limitations: “identifying behaviors of a plurality of customers who belong to the certain group in the store by analyzing the video & identifying a first behavior type that is led by a behavior of each of the customers belonging to the certain group among a plurality of behavior types that define transition of processes of the behaviors that are performed since entrance into the store until purchase of a product in the store & generating information on the purchase of the product by using the first behavior type & associating the certain group with the information on the purchase of the product”. Examiner points out Dependent Claim 2 was canceled, but Claims 9 and 15 were not canceled and are indicated as being duplicated (e.g., recited in Independent Claim 8 and Dependent Claim 9 and secondly Independent Claim 14 and Dependent Claim 15 with each of the same limitations). Examiner notes repeating the exact same limitations in the dependent claims as in the independent claims it depends on, without adding further limitations, are improper dependent claims, leading to rejections under 35 U.S.C. § 112(d) (formerly § 112, fourth paragraph) for failing to further limit the invention. These dependent claims fail its fundamental purpose of narrowing the scope of the independent claims. Examiner recommends to Applicant to cancel Dependent Claims 9 and 15 to correct this issue. Applicant may cancel the claim(s), amend the claim(s) to place the claim(s) in proper dependent form, rewrite the claim(s) in independent form, or present a sufficient showing that the dependent claim(s) complies with the statutory requirements.
Claim Rejections - 35 USC § 101
11. 35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
12. Claims 1, 4-9, 11-15 and 17-24 are rejected under 35 U.S.C. § 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claims 1, 4-9, 11-15 and 17-24 are each focused to a statutory category namely a “non-transitory computer-readable recording medium” or an “article of manufacture” (Claims 1, 4-7 and 20-24), a “method” or a “process” (Claims 8-9 and 11-13) and a “system” or an “apparatus” (Claims 14-15 and 17-19).
Step 2A Prong One: Independent Claims 1, 8 and 14 recite limitations that set forth the abstract idea(s), namely (see in bold except via strikethrough):
“” (see Independent Claim 14);
“” (see Independent Claim 14);
“” (see Independent Claim 1);
“” (see Independent Claim 8);
“identifying relationships between a plurality of customers and a sales clerk by analyzing a video in which an inside of a store is captured” (see Independent Claims 1, 8 and 14);
“identifying a plurality of customers who received customer services from the sales clerk among the plurality of customers on a basis of the identified relationships between the sales clerk and the plurality of customers” (see Independent Claims 1, 8 and 14);
“classifying each of the customers into a certain group, wherein classifying includes calculating a group similarity based on a distance between the plurality of customers, and adding a penalty value to the group similarity such that the customers who received the customer services from the sales clerk belong to different groups” (see Independent Claims 1, 8 and 14);
“identifying behaviors of a plurality of customers who belong to the certain group in the store by analyzing the video” (see Independent Claims 1, 8 and 14);
“identifying a first behavior type that is led by a behavior of each of the customers belonging to the certain group among a plurality of behavior types that define transition of processes of the behaviors that are performed since entrance into the store until purchase of a product in the store” (see Independent Claims 1, 8 and 14);
“generating information on the purchase of the product by using the first behavior type” (see Independent Claims 1, 8 and 14);
“associating the certain group with behaviors of the customers who belong to the group” (see Independent Claims 1, 8 and 14);
“associating the certain group with the information on the purchase of the product” (see Independent Claims 1, 8 and 14);
“inputting a first partial image of a first person who is extracted from the video model that is generated by inputting the first partial image of the first person extracted from the video as a feature value and adopting one of the sales clerk and the plurality of customers as a correct answer label ” (see Independent Claims 1, 8 and 14);
“determining whether the first person is one of the sales clerk and the plurality of customers” (see Independent Claims 1, 8 and 14).
Here, for Independent Claims 1, 8 and 14, the claim limitations recite the collective steps (analyzing video, identifying relationships between customers and clerks, classifying groups, and identifying behavior types for purchase generation) which are directed to the abstract idea of analyzing customer-clerk interactions to predict or generate purchase information.
Identifying relationships between customers and clerks, and identifying behaviors based on these interactions, are fundamental commercial practices. Tracking "transition of processes... since entrance until purchase" is a classic marketing and sales activity are categorized as commercial interactions and managing personal behavior or relationships between people. Additionally, the concepts such as "identifying relationships," "classifying each of the customers," and "determining whether the first person is a sales clerk" are observations and evaluations that can be performed in the human mind. The Federal Circuit has noted that simply automating these observations via "generic machine learning" does not remove them from the mental process category if the logic remains an observation of human behavior. Moreover, the step involving "calculating a group similarity based on a distance... and adding a penalty value" recites mathematical calculations. While this is a specific sub-limitation, it reinforces the abstract nature of the claim at Prong 1.
Thus, in summary, these abstract idea limitations (as identified above in bold), under their broadest reasonable interpretation of the claims as a whole, cover performance of their limitations as “Certain Methods of Organizing Human Activities” which pertains to (1) managing personal behavior or relationships or interactions between people (including teachings or following rules or instructions) or (2) commercial interactions (including marketing or sales activities or behaviors or business relations) and additionally or alternatively as “Mathematical Concepts” which pertains to (3) mathematical calculations.
Additionally, or alternatively, these abstract idea limitations (as identified above in bold), under the broadest reasonable interpretation of the claims as a whole, cover performance of their limitations as “Mental Processes” which pertains to (4) concepts performed in the human mind (including observations or evaluations or judgments) or (5) using pen and paper as a physical aid, in order to help perform these mental steps does not negate the mental nature of these limitations. The use of "physical aids" in implementing the abstract mental process, does not preclude these claims from reciting an abstract idea. See MPEP § 2106.04(a) III C.
That is, other than reciting the additional elements of (e.g., “a memory” and “a processor” & “information processing program” & “a computer”, etc…), nothing in the claim elements precludes the steps from being performed as “Certain Methods of Organizing Human Activities” which pertains to (1) managing personal behavior or relationships or interactions between people (including teachings or following rules or instructions) or (2) commercial interactions (including marketing or sales activities or behaviors or business relations) and additionally or alternatively as “Mathematical Concepts” which pertains to (3) mathematical calculations and additionally or alternatively as “Mental Processes” which pertains to (4) concepts performed in the human mind (including observations or evaluations or judgments) or (5) using pen and paper as a physical aid.
Therefore, at step 2a prong 1, Yes, Claims 1, 4-9, 11-15 and 17-24 recite an abstract idea. We proceed onto analyzing the claims at step 2a prong 2.
Step 2A Prong Two: With respect to Step 2A Prong Two of the eligibility inquiry (as explained in MPEP § 2106.04(d)), the judicial exception is not integrated into a practical application. Independent Claim 1 recites additional elements directed to: (e.g., “a memory” & “a processor” & “information processing program” & “a computer”). Independent Claim 8 recites additional elements directed to: (e.g., “a computer”). Independent Claim 14 recites additional elements directed to: (e.g., “a memory” & “a processor”). These additional elements have been considered individually and in combination, but fail to integrate the abstract idea into a practical application because they amount to using computing elements or instructions (software) to perform the abstract idea, similar to adding the words “apply it” (or an equivalent), which merely serves to link the use of the judicial exception to a particular technological environment. See MPEP § 2106.05(f) and MPEP § 2106.05(h).
Independent Claims 1, 8 and 14: Examiner notes that the additional elements of (e.g., “a machine learning model” & “neural networks”) when considered individually and as an ordered combination (as a whole), these additional elements do not integrate the abstract idea into a practical application under step 2a prong 2 due to the following: (1) the claims as a whole are limited to a particular field of use or technological environment pertaining to monitoring and analyzing identifying customers who recited customer services from the sales clerk among the plurality of customers on the basis of the identified relationships between the sales clerk and the plurality of customers using a computer in a field of customer service management of a commercial establishment (see MPEP § 2106.05 (h)) or (2) recites mere instructions to implement an abstract idea on a computer or using a computer as a tool to “apply” the recited judicial exceptions by providing the results to the user on a computer (see MPEP § 2106.05 (f)).
In addition, these limitations fail to provide an improvement to the functioning of a computer or to any other technology or technical field, fail to apply the exception with a particular machine, fail to apply the judicial exception to effect a particular treatment or prophylaxis for a disease or medical condition, fail to effect a transformation of a particular article to a different state or thing, and fail to apply/use the abstract idea in a meaningful way beyond generally linking the use of the judicial exception to a particular technological environment.
Accordingly, because the Step 2A Prong One and Prong Two analysis resulted in the conclusion that the claims are directed to an abstract idea, additional analysis under Step 2B of the eligibility inquiry must be conducted in order to determine whether any claim element or combination of elements amount to significantly more than the judicial exception. Therefore, at step 2a prong 2, Claims 1, 4-9, 11-15 and 17-24 are directed to the abstract idea and do not recite additional elements that integrate into a practical application.
Step 2B: (As explained in MPEP § 2106.05), it has been determined that the claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. Independent Claim 1 recites additional elements directed to: (e.g., “a memory” & “a processor” & “information processing program” & “a computer”). Independent Claim 8 recites additional elements directed to: (e.g., “a computer”). Independent Claim 14 recites additional elements directed to: (e.g., “a memory” & “a processor”). These elements have been considered individually and in combination, but fail to add significantly more to the claims because they amount to using computing elements or instructions (software) to perform the abstract idea, similar to adding the words “apply it” (or an equivalent), which merely serves to link the use of the judicial exception to a particular technological environment (computing environment) and does not amount to significantly more than the abstract idea itself. See MPEP § 2106.05 (h) and See MPEP § 2106.05 (f). Notably, Applicant’s Specification suggests that the claimed invention relies on nothing more than a general-purpose computer executing the instructions to implement the invention (see at least Applicant’s Specification ¶ [0031]: “The information processing apparatus 10 is, for example, an information processor, such as a desktop personal computer (PC), a notebook PC, or a server computer, which is installed inside of a store, such as a retail store, and used by a store staff, an administrator, or the like. Alternatively, the information processing apparatus 10 may be a cloud computing apparatus that is managed by a service provider who provides a cloud computing service.” and Applicant’s Specification ¶ [0134]: “The processor 10 d is a hardware circuit that reads a program that executes the same processes as each of the processing units illustrated in FIG. 2 from the HDD 10 b or the like, loads the program onto the memory 10 c, and causes processes for implementing each of the functions explained with reference to FIG. 2 or the like to work.”).
Independent Claims 1, 8 and 14: With respect to reliance on the additional elements of (e.g., “a machine learning model” & “machine learning” & “neural networks”) when considered individually and as an ordered combination (as a whole), these additional elements do not amount to significantly more than the judicial exceptions under step 2B due to the following: (1) the claims as a whole are limited to a particular field of use or technological environment pertaining to monitoring and analyzing identifying customers who recited customer services from the sales clerk among the plurality of customers on the basis of the identified relationships between the sales clerk and the plurality of customers using a computer in a field of customer service management of a commercial establishment (see MPEP § 2106.05 (h)) or (2) recites mere instructions to implement an abstract idea on a computer or using a computer as a tool to “apply” the recited judicial exceptions by providing the results to the user on a computer (see MPEP § 2106.05 (f)).
The additional elements of “machine learning” and “machine learning models” in Claims 1, 4, 8, 11, 14 and 17 does not amount to significantly more than the judicial exceptions under step 2B due to being expressly recognized as Well-Understood, Routine and Conventional (WURC) in the art.
See for example; US PG Pub (US 2020/0005331 A1) – “Information Processing Device, Terminal Device, Information Processing Method, Information Output Method, Customer Service Assistance Method, and Recording Medium”, hereinafter Warita et. al. Warita at ¶ [0065]: “The location identification unit 152 can recognize a person, using a well-known human body detection technology. For example, technologies that detect a human body or a portion (the face, a hand, or the like) of the human body included in images, using various types of image feature amounts and machine learning are generally known. Mapping of the location of a person identified by the location identification unit 152 onto the coordinate system of the map information can also be achieved using a well-known method.” Warita at ¶ [0115]: “The facing direction of a store clerk in this case may be the direction of the face of the store clerk or the direction of the line of sight of the store clerk. The location identification unit 152 can identify a facing direction of a store clerk, using a well-known face detection technology or sight line detection technology.”
The additional element of “neural networks” in Claims 1, 7-8 and 14 does not amount to significantly more than the judicial exceptions under step 2B due to being expressly recognized as Well-Understood, Routine and Conventional (WURC) in the art.
See for example; US Patent # (US 10,535,146 B1) – “Projected Image Item Tracking System”, hereinafter Buibas. Buibas at Col. 4, Lns. 51-66: “The system includes a machine learning system configured to receive the input confirming or denying that the person is associated with the motion of the item and updates the neural network based on the input. Embodiments of the invention may utilize a neural network or more generally, any type of generic function approximator. By definition the function to map inputs of before-after image pairs, or before-during-after image pairs to output actions, then the neural network can be trained to be any such function map, not just traditional convolutional neural networks, but also simpler histogram or feature based classifiers. Embodiments of the invention also enable training of the neural network, which typically involves feeding labeled data to an optimizer that modifies the network's weights and/or structure to correctly predict the labels (outputs) of the data (inputs).”
In addition, when taken as an ordered combination, the ordered combination adds nothing that is not already present as when the elements are taken individually. There is no indication that the combination of elements integrates the abstract idea into a practical application. Therefore, when viewed as a whole, these additional claim elements do not provide meaningful limitations to transform the abstract idea into a practical application of the abstract idea or that, as an ordered combination, amount to significantly more than the abstract idea itself.
Dependent Claims 4-7, 9, 11-13, 15 and 17-24 recite substantially the same or similar additional elements as addressed above and when considered individually and as an ordered combination (as a whole) with these limitations recite the same abstract idea(s) as shown in Independent Claims 1, 8 and 14 along with further steps/details pertaining to “Certain Methods of Organizing Human Activities” which pertains to (1) managing personal behavior or relationships or interactions between people (including teachings or following rules or instructions) or (2) commercial interactions (including marketing or sales activities or behaviors or business relations) and additionally or alternatively as “Mathematical Concepts” which pertains to (3) mathematical calculations and additionally or alternatively as “Mental Processes” which pertains to (4) concepts performed in the human mind (including observations or evaluations or judgments) or (5) using pen and paper as a physical aid.
Dependent Claims 5-6, 9, 12-13, 15 and 18-23 further narrow the abstract ideas, and are therefore still ineligible for the reasons previously provided in Steps 2A Prong 2 and 2B for Independent Claims 1, 8 and 14. Dependent Claims 4, 7, 11, 17 and 24: With respect to reliance on (e.g., “machine learning models” (see Dependent Claims 4, 11 and 17) & “neural networks” (see Dependent Claim 7) & “Human Object Interaction Detection (HOID) model” (see Dependent Claim 24)) as additional elements shown in Dependent Claims 4, 7, 11, 17 and 24 when considered individually and as a whole in view of these claim limitations, these additional elements do not provide limitations that are indicative of integration into a practical application under step 2a prong 2 and also do not recite additional elements that amount to significantly more than the recited judicial exceptions under step 2B due to: (1) the claims as a whole are limited to a particular field of use or technological environment pertaining to monitoring and analyzing identifying customers who recited customer services from the sales clerk among the plurality of customers on the basis of the identified relationships between the sales clerk and the plurality of customers using a computer in a field of customer service management of a commercial establishment (see MPEP § 2106.05 (h)) or (2) recites mere instructions to implement an abstract idea on a computer or using a computer as a tool to “apply” the recited judicial exceptions by providing the results to the user on a computer (see MPEP § 2106.05 (f)).
Therefore, these claims do not amount to “significantly more” than the abstract idea because they neither (1) recite any improvements to another technology or technical field; (2) recite any improvements to the functioning of the computer itself; (3) apply the judicial exception with, or by use of, a particular machine; (4) effect a transformation or reduction of a particular article to a different state or thing; (5) add a specific limitation other than what is well-understood, routine and conventional in the field; (6) add unconventional steps that confine the claim to a particular useful application; nor (7) provide other meaningful limitations beyond generally linking the use of the judicial exception to a particular technological environment.
The ordered combination of elements in the Dependent Claims (including the limitations inherited from the parent claim(s)) add nothing that is not already present as when the elements are taken individually. There is no indication that the combination of elements improves the functioning of a computer or improves any other technology. Accordingly, the subject matter encompassed by the dependent claims fails to amount to a practical application or significantly more than the abstract idea itself. Therefore, under Step 2B, Claims 1, 4-9, 11-15 and 17-24 do not include additional elements that are sufficient to amount to significantly more than the recited judicial exceptions. Thus, Claims 1, 4-9, 11-15 and 17-24 are ineligible with respect to the 35 U.S.C. § 101 analysis.
Claim Rejections - 35 USC § 103
13. This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
14. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
15. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
16. Claims 1, 4, 6-9, 11, 13-15, 17, 19 and 21-22 are rejected under 35 U.S.C. 103 as being unpatentable over US Patent # (US 7,957,565 B1) hereinafter Sharma, et. al., in view of US PG Pub (US 2019/0205965 A1) hereinafter Li, et. al., in view of US PG Pub (US 2021/0161451 A1) hereinafter Crisfalusi, et. al., and in further view of US PG Pub (US 2022/0067689 A1) hereinafter Guack, et. al.
Regarding Independent Claims 1 and 8, Sharma non-transitory computer-readable medium / method for customer service purchasing behavior between the sales clerk and customer teaches the following:
- having stored therein an information processing program that causes a computer to execute a process, the process comprising (see at least Sharma: Fig. 3 & Col. 10, Lns. 26-46. Sharma teaches that the means for control and processing can be a general-purpose personal computer, such as a Pentium 4 PC, or a dedicated hardware that can carry out the required computation.);
- identifying relationships between a plurality of customers and a sales clerk by analyzing a video in which an inside of a store is captured (see at least Sharma: Fig. 7 & Col. 8, Lns. 32-39. Sharma teaches that the employees' influence on the customers' activities in the retail space can also be measured by analyzing the relationship between the employee data and the customer behavior analysis in the retail space. The solution leverages the strengths of the technologies in the present invention and processes to deliver a new level of access to the understanding of customer activities, not to mention the employees' activities. Sharma also teaches at Col. 11, Lns. 36-41: “Rules for utilizing the relationship among the tracks of people, in which the relationship shows the employee and non-employee interaction. Examples of such relationship can comprise one-to-many mapping of tracking interaction over a period of time, in which one track interacts with multiple tracks.” Sharma also teaches at Col. 13, Lns. 28-37: Sharma notes captures a plurality of input images, including “video input images 1331 through “video input images N’ 333, of the persons in the physical space 130 by a plurality of means for capturing images 100. The plurality of input images for person detection 710 and person tracking 714. The person detection 710 and person tracking 714 are performed for each person in the plurality of persons in each field of view of the plurality of means for capturing images 100.);
- identifying a plurality of customers who received customer services (see at least Sharma: Col. 2, Lns. 6-39. Sharma teaches method for identifying activity of customers for a marketing purpose or activity of objects in a surveillance area, by comparing the detected objects with the graphs from a database.) from the sales clerk among the plurality of customers on a basis of the identified relationships between the sales clerk and the plurality of customers (see at least Sharma: Figs. 7-10 & Col. 8, Lns. 32-36 & Col. 15, Lns. 19-29. Sharma notes that the employees' influence on the customers' activities in the retail space can also be measured by analyzing the relationship between the employee data and the customer behavior analysis in the retail space. See also Sharma at Col. 5, Lns. 38-47: A deep understanding of customer-only behaviors, design of effective retail programs, efficient product assortment strategies, efficient use of space, improvement in the performance and productivity of employees through employee behavior statistics, increased customer service performance through refined employees' interaction with customers, and actual measurement for the correlation between the employees' performance and sales. See also Sharma at Col. 11, Lns. 29-41: Rules for utilizing the relationship among the tracks of people, in which the relationship shows the employee and non-employee interaction. Examples of such relationship can comprise one-to-many mapping of tracking interaction over a period of time, in which one track interacts with multiple tracks. See also Sharma at Col. 15, Lns. 19-29: Sharma teaches that based on the observation on the people trajectories, multiple high-level behavioral classes are defined, such as stocking, helping customers, or entering/leaving employee area.).
Moreover, Sharma non-transitory computer-readable medium / method for customer service purchasing behavior between the sales clerk and customer does not explicitly disclose, but Li in the analogous art for customer service purchasing behavior between the sales clerk and customer teaches the following limitations:
- classifying each of the customers into a certain group (see at least Li: Fig. 7 & ¶ [0016] & ¶ [0074]. Li teaches classifying customers into a plurality of customer clusters based on the identifier of the customer, and determining an identifier of a corresponding category based on per-category purchase data regarding each of the plurality of customer clusters. See also Li at ¶ [0074]: The customer item recommending apparatus obtains personal information such as an age, a gender and an ethnic group of the customer by analyzing feature vectors 361 with respect to the customer image 310. For example, the customer item recommending apparatus extracts the feature vectors 361 from the customer image 310 based on an image recognition model and identifies the age, the gender, and the ethnic group of the customer corresponding to the extracted feature vectors 361.), wherein classifying includes calculating a group similarity based on a distance between the plurality of customers (see at least Li: Fig. 7 & ¶ [0023] & ¶ [0083] & ¶ [0162]. Li notes that the server 1310 estimates a position of the customer in the physical store and transmits item information related to the recommended item to an additional display 1320 within a threshold distance from the estimated position of the customer. See also Li at ¶ [0023]: “The providing may include transmitting item information related to the recommended item to a display disposed within a threshold distance from a position of the customer”. See also Li at ¶ [0060]: Further, when customer identification using a camera is not available due to any one or any combination of an FOV, a resolution, a distance. See also Li at ¶ [0083]: The customer item recommending apparatus calculates a similarity level between the customers based on an inner product between the customer feature vectors of the individual customers. The customer item recommending apparatus classifies the customers into individual customer clusters based on the similarity level between the customers. The individual customer clusters are clusters each including customers having similar purchase histories.), and adding a penalty value to the group similarity such that the customers who received the customer services from the sales clerk belong to different groups (see at least Li: Fig. 7 & ¶ [0016] & ¶ [0074] & ¶ [0139]. Li notes that the customer item recommending apparatus obtains embedding vectors fq, fp, and fn by mapping each of the images Iq, Ip, and In to an embedding layer. The loss function Loss of Equation 1 has two penalties. Lpos denotes a function that penalizes a positive pair that is too far apart, and Lneg denotes a function that penalizes a negative pair that is too closer than a margin m. StyleSim(fp, fq) denotes a function that calculates a similarity, for example, a vector inner product, between fp and fq, and StyleSim(fn, fq) denotes a function that calculates a similarity between fn and fq. See also Li at ¶ [0174] & Fig. 14. The customer item recommending apparatus provides a convenient shopping service to the customer. See also Li at ¶ [0192]: The customer item recommending apparatus 1600 provides personalized recommendation information with respect to related product and service to the customer with high quality in an AR shopping application based on a purchase tendency identified with respect to the individual customer.).
It would have been obvious for one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the teachings of Sharma non-transitory computer-readable medium / method for customer service purchasing behavior between the sales clerk and customer with the aforementioned teachings of: classifying each of the customers into a certain group, wherein classifying includes calculating a group similarity based on a distance between the plurality of customers, and adding a penalty value to the group similarity such that the customers who received the customer services from the sales clerk belong to different groups, in view of Li, whereby the online procedure includes an operation of acquiring a customer image and an operation of automatically determining the purchase tendency model most suitable for the corresponding customer from the purchase tendency model database based on a visual feature of a customer appearance identified from the customer image (see at least Li: ¶ [0068]). Additionally, the customer item recommending apparatus calculates a similarity level between the customers based on an inner product between the customer feature vectors of the individual customers. The customer item recommending apparatus classifies the customers into individual customer clusters based on the similarity level between the customers. The individual customer clusters are clusters each including customers having similar purchase histories (see at least Li: ¶ [0083].)
Further, the claimed invention is merely a combination of old elements in a similar field for customer service purchasing behavior between the sales clerk and customer, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that, given the existing technical ability to combine the elements as evidenced by Li, the results of the combination were predictable.
Moreover, Sharma / Li non-transitory computer-readable medium / method for customer service purchasing behavior between the sales clerk and customer does not explicitly disclose, but Crisfalusi in the analogous art for customer service purchasing behavior between the sales clerk and customer teaches the following limitations:
- identifying behaviors of a plurality of customers who belong to the certain group in the store by analyzing the video (see at least Crisfalusi: Fig. 2 & ¶ [0021] & ¶ [0058] & ¶ [0075]. Crisfalusi notes that the environment 100 includes a retail store area 102, an image capturing component 104 for capturing image and video data pertaining to the retail store area 102 in real time, and a central control unit 106 communicatively coupled to various computing devices present in the retail store area 102, and to the image capturing component 104 for classifying and studying the behaviour of the customers in the retail store area 102. See also Crisfalusi at FIG. 2 which notes the social behavioral space 200 that illustrates four different types of social attribute classes amongst humans, depending on the distribution of social behaviors within the social behavioral space 200. See also Crisfalusi at ¶ [0058]: The real-time labelling of social attribute class further assists the retailer with real time interpretation of the effect of changing a customer's local environment from an individual customer perspective or a collective customer group-wise perspective. The real-time labelling of social attribute class further facilitates the retailer with rapid optimization of the customer's environment to maximize the sales outcome. See also Crisfalusi at ¶ [0075]: A social attribute class is assigned to the human subject from a group consisting of ‘driver,’ ‘analytical,’ ‘amiable’ and ‘expressive’. In an embodiment of the present disclosure, a set of class scores is computed for the human subject, wherein the set includes first, second and third class scores for first, second and third social attributes. See also Crisfalusi at Fig. 7.);
- identifying a first behavior type that is led by a behavior of each of the customers belonging to the certain group among a plurality of behavior types that define transition of processes of the behaviors (see at least Crisfalusi: Figs. 2-3 & ¶ [0032] & ¶ [0036]. Crisfalusi notes that the pose detection component 306 may include convolutional neural networks (CNN) and dedicated logic elements for assessing the movements of the subject's joints over time, and detecting patterns associated with specific poses and tracking the subject's pose movements over time, for example, for distinguishing between right and left hands, and linking the joints according to a skeletal frame or sequences of joint movements associated with the adoption of specific poses. See also Crisfalusi at ¶ [0027-0028] & Fig. 2. Crisfalusi teaches that the first social attribute class 202 may be referred to as ‘Driver’, according to which a person is extroverted and task focused; and is characterized as being objective, focused, decisive, high energy and fast paced. The second social attribute class 204 may be referred to as ‘Analytical’ according to which a person is introverted and task focused; and is characterized as being fact and task oriented, logical, requiring evidence such as benchmarks and statistics, and needing time to decide. The third social attribute class 206 may be referred to as ‘Amiable’ according to which a person is introverted and people focused; and is characterized as consistent, dependable and stabilizing, cooperative and easy to get along with, indecisive and risk-averse. The fourth social attribute class 208 may be referred to as ‘Expressive’ according to which a person is extroverted and people focused; and is characterized as being relationship-oriented, verbally and socially adept, status conscious and a big-picture thinker. See also Crisfalusi at ¶ [0036]: “The social attribute classification component 312 is configured to observe behavioral traits using Artificial Intelligence (AI) to map out a behavioral type, and to personalize human interactive experience based on their behavioral type”.) that are performed since entrance into the store until purchase of a product in the store (see at least Crisfalusi: Fig. 5 & ¶ [0037] & ¶ [0062-0064]. Crisfalusi teaches that the interaction time attribute (Tint) is the average time spent by the customers 108 interacting with people or responding to a trigger, for example, in-store displayed adverts or alternative forced routes in the store or people welcoming customers at the entrance to the store; and the analysis time attribute (Tanal) includes the average time spent by the customers 108 in analyzing a product. See also Crisfalusi at ¶ [0024]: The scanning zone is an area in front of the scanner where the user brings up the items for scanning for the purpose of buying of those items. See also Crisfalusi at ¶ [0062]: The system 506 may assign a unique ID to each customer upon their entry in the retail store area 502 that may be maintained over the entire duration of the customer's stay in the retail store area 502, for example, during the customer's entire journey from the entrance of the store to exit from targeted area(s) of the store. The social attribute classes assigned to a customer may be computed at any given time using the entire history of attribute values observed up to that point in the customer's journey. If there are specific places with advertising panels in the store, the customer's social attribute classification may be determined every time the customer reaches a specific location in front of such panel. See also Crisfalusi at ¶ [0062]: The system 506 may be configured to maintain consistent labelling of the customer during their stay in the store by detecting when a given customer enters the field of view of proximal cameras, using crude pattern recognition identification approach, and face recognition approach. The system 506 may assign a unique ID to each customer upon their entry in the retail store area 502 that may be maintained over the entire duration of the customer's stay in the retail store area 502, for example, during the customer's entire journey from the entrance of the store to exit from targeted area(s) of the store. The social attribute classes assigned to a customer may be computed at any given time using the entire history of attribute values observed up to that point in the customer's journey.)
- generating information on the purchase of the product by using the first behavior type (see at least Crisfalusi: ¶ [0022] & ¶ [0040-0048] & ¶ [0058]. Crisfalusi notes that the retail store area 102 includes first through sixth customers 108 a, 108 b, 108 c, 108 d, 108 e, and 108 f that are configured to store items for purchase by the customers 108, to enable the customers 108 to bill their products. See also Crisfalusi at ¶ [0036] & Fig. 3: The social attribute classification component 312 is configured to observe behavioral traits using Artificial Intelligence (AI) to map out a behavioral type, and to personalize human interactive experience based on their behavioral type. See also Crisfalusi at ¶ [0048]: Engagement with a shop assistant to obtain directions to a particular product may indicate a different intention to obtain the product than interactions with accompanying persons, such as family members of friends. It may be easy to distinguish shop assistants from family members if shop assistants wear a uniform or if other persons are wearing clothing which allows them to be readily distinguished from persons accompanying the subject. See also Crisfalusi at ¶ [0048]: The action detection component 308, and the activity detection component 310, with one or more object recognition algorithms designed to distinguish between the different objects that may be held in a subject's hands. For example, distinguishing between a sales product, and the user's mobile phone. With this distinction, the time spent by the subject with items other than retail products in their hands is excluded. See also Crisfalusi at ¶ [0058]: The real-time labelling of social attribute class further assists the retailer with real time interpretation of the effect of changing a customer's local environment from an individual customer perspective or a collective customer group-wise perspective. The real-time labelling of social attribute class further facilitates the retailer with rapid optimization of the customer's environment to maximize the sales outcome. See also Crisfalusi at Fig. 5.)
- associating the certain group with behaviors of the customers who belong to the group (see at least Crisfalusi: Fig. 7 & ¶ [0032-0034] & ¶ [0069-0071]. Crisfalusi notes that the pose detection component may include convolutional neural networks (CNN) and dedicated logic elements for assessing the movements of the subject's joints over time, and detecting patterns associated with specific poses and tracking the subject's pose movements over time, for example, for distinguishing between right and left hands, and linking the joints according to a skeletal frame or sequences of joint movements associated with the adoption of specific poses. See also Crisfalusi at ¶ [0034]: The activity detection component 310 is configured to detect the presence of an observed sequence of actions determined by the action detection component 308 associated with the performance of a pre-defined task. In an example, item selection activity may include multiple actions such as picking up an item, moving item to shopping basket/trolley, and putting the item in the shopping basket/trolley. See also Crisfalusi at ¶ [0039-0040]: The process of aggregating may include evaluating particular actions separately from others, for example, weighting joint speeds associated with walking differently from hand movements in moving items on shelves. See also Crisfalusi at ¶ [0069-0071]: The activity may be detected by detecting the presence of an observed sequence of actions associated with the performance of a pre-defined task. In an example, item selection activity may include multiple actions such as picking up an item, moving item to shopping basket/trolley, and putting the item in the shopping basket/trolley.);
- associating the certain group with the information on the purchase of the product (see at least Crisfalusi: ¶ [0022] & ¶ [0058] & ¶ [0075]. Crisfalusi notes that the real-time labelling of social attribute class further assists the retailer with real time interpretation of the effect of changing a customer's local environment from an individual customer perspective or a collective customer group-wise perspective. The real-time labelling of social attribute class further facilitates the retailer with rapid optimization of the customer's environment to maximize the sales outcome. See also Crisfalusi at ¶ [0075]: At 716, a social attribute class is automatically assigned to the human subject based on the values of the first, second and third social attributes. A social attribute class is assigned to the human subject from a group consisting of ‘driver,’ ‘analytical,’ ‘amiable’ and ‘expressive’. A set of class scores is computed for the human subject, wherein the set includes first, second and third class scores for first, second and third social attributes. See also Crisfalusi at Figs. 2-5.).
It would have been obvious for one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the teachings of Sharma / Li non-transitory computer-readable medium / method for customer service purchasing behavior between the sales clerk and customer with the aforementioned teachings of: identifying behaviors of a plurality of customers who belong to the certain group in the store by analyzing the video & identifying a first behavior type that is led by a behavior of each of the customers belonging to the certain group among a plurality of behavior types that define transition of processes of the behaviors that are performed since entrance into the store until purchase of a product in the store & generating information on the purchase of the product by using the first behavior type & associating the certain group with behaviors of the customers who belong to the group & associating the certain group with the information on the purchase of the product, and in further view of Crisfalusi, whereby the real-time labelling of social attribute class enables the retailer to obtain an evolving insight into the customer's social attribute classification and the effect of the customer's interactions with the other related or unrelated individuals in the same field of view. The real-time labelling of social attribute class further assists the retailer with real time interpretation of the effect of changing a customer's local environment from an individual customer perspective or a collective customer group-wise perspective. The real-time labelling of social attribute class further facilitates the retailer with rapid optimization of the customer's environment to maximize the sales outcome (see at least Crisfalusi: ¶ [0058].)
Further, the claimed invention is merely a combination of old elements in a similar field for customer service purchasing behavior between the sales clerk and customer, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that, given the existing technical ability to combine the elements as evidenced by Crisfalusi, the results of the combination were predictable.
Moreover, Sharma / Li / Crisfalusi non-transitory computer-readable medium / method for customer service purchasing behavior between the sales clerk and customer does not explicitly disclose, but Guack in the analogous art for customer service purchasing behavior between the sales clerk and customer teaches the following limitations:
- inputting (see at least Guack: ¶ [0078]: “The input interface 120 may include a camera for inputting a video signal, a microphone for receiving an audio signal, and a user input interface for receiving information from a user.”) a first partial image of a first person who is extracted from the video into a machine learning model (see at least Guack: ¶ [0219] & ¶ [0245] & ¶ [0266-0267]. Guack notes that “the tracker can handle partial occlusion or complete occlusions for certain time periods, as described in further detail below in FIG. 12.” See also Guack at ¶ [0245]: The monocular cameras 1303 a, 1303 b manage partial and complete occlusions of users and shelves better than a single depth camera. See also Guack at ¶ [0266-0267]: The output layer 1426 is a task-dependent layer that performs a “computer vision related task such as feature extraction, object recognition, object detection, pose estimation, or the like. Neural networks, such as CNNs are often used to solve computer vision problems including feature extraction, object recognition, object detection, and pose estimation.” See also Guack at ¶ [0310]) that is generated using a partial image of a person extracted from the video as a feature value (see at least Guack: ¶ [0079-0080] & ¶ [0245]. Guack notes that “The learned artificial neural network may be referred to as a learning model. The learning model may be used to an infer result value for new input data rather than learning data, and the inferred value may be used as a basis for determination to perform a certain operation.” The processor 180 or the learning processor 130 may extract an input feature by preprocessing the input data. See also Guack at ¶ [0245]: Guack notes that “The monocular cameras 1303 a, 1303 b manage partial and complete occlusions of users and shelves better than a single depth camera.) and adopting one of the sales clerk and the customer as a correct answer label (see at least Guack: ¶ [0057] & ¶ [0112] & ¶ [0310-0311]. Guack teaches that “the supervised learning may refer to a method of learning an artificial neural network in a state in which a label for learning data is given, and the label may mean the correct answer (or result value) that the artificial neural network must infer when the learning data is input to the artificial neural network. The unsupervised learning may refer to a method of learning an artificial neural network in a state in which a label for learning data is not given. Supervised learning may refer to a method for training an artificial neural network with training data that has been given a label. In addition, the label may refer to a target answer (or a result value) to be guessed by the artificial neural network when the training data is inputted to the artificial neural network. See also Guack at ¶ [0310-0311]: “The association method 1700 b is performed as an initial step when a clerk is stocking a bin on a shelf unit with items. The association method 1700 b provides an initial set of clustering by providing the machine learning model with specific unit weights associated with specific items identified by SKU numbers. For example, the method in block 1701 b may be performed by a store clerk when he is initially stocking a shelf unit with items. The store clerk has a mapping of items, a mapping of the items and their SKU numbers, and a corresponding bin on a shelf unit that the item should be placed in. See also Guack at ¶ [0317]: The training method 1700 c is performed by a store clerk when he is initially stocking a shelf unit (e.g., shelf unit 503 shown in FIGS. 5-7) with items in their corresponding bins on the shelf unit. The training method 1700 c is performed by a store clerk when the store clerk is moving an item from one bin on the shelf unit to another bin on the shelf unit. See also Guack at ¶ [0389]: The module fusion approach finds correspondence between two or more modules, assigns unique ID to people and objects, and corrects the detections.) into a neural network (see at least Guack: ¶ [0052-0057] & ¶ [0201].)
- determine whether the first person is one of the sales clerk and the plurality of customers (see at least Guack: ¶ [0192-0194] & ¶ [0317] & Figs. 6-7. Guack teaches that a shopper 501 (e.g., a first user), a shelf unit 503, an item 505, a retail store employee 507 (e.g., a second user), a retailer application platform 509, and at least one camera 511 (e.g., image sensor). The operating environment 500 is configured to provide shoppers 501 (or users) with an automated shopping experience where shoppers can purchase items without having to go through a self-check-out line, interact with retail store employees 507, or interact with a cashier. See also Guack at [0317]: Training method 1700 c is performed by a store clerk when he is initially stocking a shelf unit (e.g., shelf unit 503 shown in FIGS. 5-7) with items in their corresponding bins on the shelf unit. The training method 1700 c is performed by a store clerk when the store clerk is moving an item from one bin on the shelf unit to another bin on the shelf unit. See also Guack at ¶ [0310-0311].)
It would have been obvious for one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the teachings of Sharma / Li / Crisfalusi non-transitory computer-readable medium / method for customer service purchasing behavior between the sales clerk and customer with the aforementioned teachings of: inputting a first partial image of a first person who is extracted from the video into a ML model that is generated through ML using a partial image of a person extracted from the video as a feature value and adopting one of the sales clerk and the customer as a correct answer label into a neural network and determine whether the first person is one of the sales clerk and the plurality of customers, and in further view of Guack, wherein the supervised learning may refer to a method of learning an artificial neural network in a state in which a label for learning data is given, and the label may mean the correct answer (or result value) that the artificial neural network must infer when the learning data is input to the artificial neural network (see at least Guack: ¶ [0057]).
Further, the claimed invention is merely a combination of old elements in a similar field for customer service purchasing behavior between the sales clerk and customer, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that, given the existing technical ability to combine the elements as evidenced by Guack, the results of the combination were predictable.
Regarding Independent Claim 14, Sharma information processing apparatus for customer service purchasing behavior between the sales clerk and customer teaches the following:
- a memory (see at least Sharma: Fig. 3 & Col. 10, Lns. 61-63.);
- a processor (see at least Sharma: Fig. 3 & Col. 10, Lns. 56-61.) coupled to the memory (see at least Sharma: Fig. 3 & Col. 10, Lns. 61-63.) and the processor (see at least Sharma: Fig. 3 & Col. 10, Lns. 56-61.) configured to:
- identifying relationships between a plurality of customers and a sales clerk by analyzing a video in which an inside of a store is captured (see at least Sharma: Fig. 7 & Col. 8, Lns. 32-39. Sharma teaches that the employees' influence on the customers' activities in the retail space can also be measured by analyzing the relationship between the employee data and the customer behavior analysis in the retail space. The solution leverages the strengths of the technologies in the present invention and processes to deliver a new level of access to the understanding of customer activities, not to mention the employees' activities. Sharma also teaches at Col. 11, Lns. 36-41: “Rules for utilizing the relationship among the tracks of people, in which the relationship shows the employee and non-employee interaction. Examples of such relationship can comprise one-to-many mapping of tracking interaction over a period of time, in which one track interacts with multiple tracks.” Sharma also teaches at Col. 13, Lns. 28-37: Sharma notes captures a plurality of input images, including “video input images 1331 through “video input images N’ 333, of the persons in the physical space 130 by a plurality of means for capturing images 100. The plurality of input images for person detection 710 and person tracking 714. The person detection 710 and person tracking 714 are performed for each person in the plurality of persons in each field of view of the plurality of means for capturing images 100.).
- identifying a plurality of customers who received customer services (see at least Sharma: Col. 2, Lns. 6-39. Sharma teaches method for identifying activity of customers for a marketing purpose or activity of objects in a surveillance area, by comparing the detected objects with the graphs from a database.) from the sales clerk among the plurality of customers on a basis of the identified relationships between the sales clerk and the plurality of customers (see at least Sharma: Figs. 7-10 & Col. 8, Lns. 32-36 & Col. 15, Lns. 19-29. Sharma notes that the employees' influence on the customers' activities in the retail space can also be measured by analyzing the relationship between the employee data and the customer behavior analysis in the retail space. See also Sharma at Col. 5, Lns. 38-47: A deep understanding of customer-only behaviors, design of effective retail programs, efficient product assortment strategies, efficient use of space, improvement in the performance and productivity of employees through employee behavior statistics, increased customer service performance through refined employees' interaction with customers, and actual measurement for the correlation between the employees' performance and sales. See also Sharma at Col. 11, Lns. 29-41: Rules for utilizing the relationship among the tracks of people, in which the relationship shows the employee and non-employee interaction. Examples of such relationship can comprise one-to-many mapping of tracking interaction over a period of time, in which one track interacts with multiple tracks. See also Sharma at Col. 15, Lns. 19-29: Sharma teaches that based on the observation on the people trajectories, multiple high-level behavioral classes are defined, such as stocking, helping customers, or entering/leaving employee area.).
Moreover, Sharma information processing apparatus for customer service purchasing behavior between the sales clerk and customer does not explicitly disclose, but Li in the analogous art for customer service purchasing behavior between the sales clerk and customer teaches the following limitations:
- classifying each of the customers into a certain group (see at least Li: Fig. 7 & ¶ [0016] & ¶ [0074]. Li teaches classifying customers into a plurality of customer clusters based on the identifier of the customer, and determining an identifier of a corresponding category based on per-category purchase data regarding each of the plurality of customer clusters. See also Li at ¶ [0074]: The customer item recommending apparatus obtains personal information such as an age, a gender and an ethnic group of the customer by analyzing feature vectors 361 with respect to the customer image 310. For example, the customer item recommending apparatus extracts the feature vectors 361 from the customer image 310 based on an image recognition model and identifies the age, the gender, and the ethnic group of the customer corresponding to the extracted feature vectors 361.), wherein classifying includes calculating a group similarity based on a distance between the plurality of customers (see at least Li: Fig. 7 & ¶ [0023] & ¶ [0083] & ¶ [0162]. Li notes that the server 1310 estimates a position of the customer in the physical store and transmits item information related to the recommended item to an additional display 1320 within a threshold distance from the estimated position of the customer. See also Li at ¶ [0023]: “The providing may include transmitting item information related to the recommended item to a display disposed within a threshold distance from a position of the customer”. See also Li at ¶ [0060]: Further, when customer identification using a camera is not available due to any one or any combination of an FOV, a resolution, a distance. See also Li at ¶ [0083]: The customer item recommending apparatus calculates a similarity level between the customers based on an inner product between the customer feature vectors of the individual customers. The customer item recommending apparatus classifies the customers into individual customer clusters based on the similarity level between the customers. The individual customer clusters are clusters each including customers having similar purchase histories.), and adding a penalty value to the group similarity such that the customers who received the customer services from the sales clerk belong to different groups (see at least Li: Fig. 7 & ¶ [0016] & ¶ [0074] & ¶ [0139]. Li notes that the customer item recommending apparatus obtains embedding vectors fq, fp, and fn by mapping each of the images Iq, Ip, and In to an embedding layer. The loss function Loss of Equation 1 has two penalties. Lpos denotes a function that penalizes a positive pair that is too far apart, and Lneg denotes a function that penalizes a negative pair that is too closer than a margin m. StyleSim(fp, fq) denotes a function that calculates a similarity, for example, a vector inner product, between fp and fq, and StyleSim(fn, fq) denotes a function that calculates a similarity between fn and fq. See also Li at ¶ [0174] & Fig. 14. The customer item recommending apparatus provides a convenient shopping service to the customer. See also Li at ¶ [0192]: The customer item recommending apparatus 1600 provides personalized recommendation information with respect to related product and service to the customer with high quality in an AR shopping application based on a purchase tendency identified with respect to the individual customer.).
It would have been obvious for one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the teachings of Sharma information processing apparatus for customer service purchasing behavior between the sales clerk and customer with the aforementioned teachings of: classifying each of the customers into a certain group, wherein classifying includes calculating a group similarity based on a distance between the plurality of customers, and adding a penalty value to the group similarity such that the customers who received the customer services from the sales clerk belong to different groups, in view of Li, whereby the online procedure includes an operation of acquiring a customer image and an operation of automatically determining the purchase tendency model most suitable for the corresponding customer from the purchase tendency model database based on a visual feature of a customer appearance identified from the customer image (see at least Li: ¶ [0068]). Additionally, the customer item recommending apparatus calculates a similarity level between the customers based on an inner product between the customer feature vectors of the individual customers. The customer item recommending apparatus classifies the customers into individual customer clusters based on the similarity level between the customers. The individual customer clusters are clusters each including customers having similar purchase histories (see at least Li: ¶ [0083].)
Further, the claimed invention is merely a combination of old elements in a similar field for customer service purchasing behavior between the sales clerk and customer, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that, given the existing technical ability to combine the elements as evidenced by Li, the results of the combination were predictable.
Moreover, Sharma / Li information processing apparatus for customer service purchasing behavior between the sales clerk and customer does not explicitly disclose, but Crisfalusi in the analogous art for customer service purchasing behavior between the sales clerk and customer teaches the following limitations:
- identifying behaviors of a plurality of customers who belong to the certain group in the store by analyzing the video (see at least Crisfalusi: Fig. 2 & ¶ [0021] & ¶ [0058] & ¶ [0075]. Crisfalusi notes that the environment 100 includes a retail store area 102, an image capturing component 104 for capturing image and video data pertaining to the retail store area 102 in real time, and a central control unit 106 communicatively coupled to various computing devices present in the retail store area 102, and to the image capturing component 104 for classifying and studying the behaviour of the customers in the retail store area 102. See also Crisfalusi at FIG. 2 which notes the social behavioral space 200 that illustrates four different types of social attribute classes amongst humans, depending on the distribution of social behaviors within the social behavioral space 200. See also Crisfalusi at ¶ [0058]: The real-time labelling of social attribute class further assists the retailer with real time interpretation of the effect of changing a customer's local environment from an individual customer perspective or a collective customer group-wise perspective. The real-time labelling of social attribute class further facilitates the retailer with rapid optimization of the customer's environment to maximize the sales outcome. See also Crisfalusi at ¶ [0075]: A social attribute class is assigned to the human subject from a group consisting of ‘driver,’ ‘analytical,’ ‘amiable’ and ‘expressive’. In an embodiment of the present disclosure, a set of class scores is computed for the human subject, wherein the set includes first, second and third class scores for first, second and third social attributes. See also Crisfalusi at Fig. 7.);
- identifying a first behavior type that is led by a behavior of each of the customers belonging to the certain group among a plurality of behavior types that define transition of processes of the behaviors (see at least Crisfalusi: Figs. 2-3 & ¶ [0032] & ¶ [0036]. Crisfalusi notes that the pose detection component 306 may include convolutional neural networks (CNN) and dedicated logic elements for assessing the movements of the subject's joints over time, and detecting patterns associated with specific poses and tracking the subject's pose movements over time, for example, for distinguishing between right and left hands, and linking the joints according to a skeletal frame or sequences of joint movements associated with the adoption of specific poses. See also Crisfalusi at ¶ [0027-0028] & Fig. 2. Crisfalusi teaches that the first social attribute class 202 may be referred to as ‘Driver’, according to which a person is extroverted and task focused; and is characterized as being objective, focused, decisive, high energy and fast paced. The second social attribute class 204 may be referred to as ‘Analytical’ according to which a person is introverted and task focused; and is characterized as being fact and task oriented, logical, requiring evidence such as benchmarks and statistics, and needing time to decide. The third social attribute class 206 may be referred to as ‘Amiable’ according to which a person is introverted and people focused; and is characterized as consistent, dependable and stabilizing, cooperative and easy to get along with, indecisive and risk-averse. The fourth social attribute class 208 may be referred to as ‘Expressive’ according to which a person is extroverted and people focused; and is characterized as being relationship-oriented, verbally and socially adept, status conscious and a big-picture thinker. See also Crisfalusi at ¶ [0036]: “The social attribute classification component 312 is configured to observe behavioral traits using Artificial Intelligence (AI) to map out a behavioral type, and to personalize human interactive experience based on their behavioral type”.) that are performed since entrance into the store until purchase of a product in the store (see at least Crisfalusi: Fig. 5 & ¶ [0037] & ¶ [0062-0064]. Crisfalusi teaches that the interaction time attribute (Tint) is the average time spent by the customers 108 interacting with people or responding to a trigger, for example, in-store displayed adverts or alternative forced routes in the store or people welcoming customers at the entrance to the store; and the analysis time attribute (Tanal) includes the average time spent by the customers 108 in analyzing a product. See also Crisfalusi at ¶ [0024]: The scanning zone is an area in front of the scanner where the user brings up the items for scanning for the purpose of buying of those items. See also Crisfalusi at ¶ [0062]: The system 506 may assign a unique ID to each customer upon their entry in the retail store area 502 that may be maintained over the entire duration of the customer's stay in the retail store area 502, for example, during the customer's entire journey from the entrance of the store to exit from targeted area(s) of the store. The social attribute classes assigned to a customer may be computed at any given time using the entire history of attribute values observed up to that point in the customer's journey. If there are specific places with advertising panels in the store, the customer's social attribute classification may be determined every time the customer reaches a specific location in front of such panel. See also Crisfalusi at ¶ [0062]: The system 506 may be configured to maintain consistent labelling of the customer during their stay in the store by detecting when a given customer enters the field of view of proximal cameras, using crude pattern recognition identification approach, and face recognition approach. The system 506 may assign a unique ID to each customer upon their entry in the retail store area 502 that may be maintained over the entire duration of the customer's stay in the retail store area 502, for example, during the customer's entire journey from the entrance of the store to exit from targeted area(s) of the store. The social attribute classes assigned to a customer may be computed at any given time using the entire history of attribute values observed up to that point in the customer's journey.)
- generating information on the purchase of the product by using the first behavior type (see at least Crisfalusi: ¶ [0022] & ¶ [0040-0048] & ¶ [0058]. Crisfalusi notes that the retail store area 102 includes first through sixth customers 108 a, 108 b, 108 c, 108 d, 108 e, and 108 f that are configured to store items for purchase by the customers 108, to enable the customers 108 to bill their products. See also Crisfalusi at ¶ [0036] & Fig. 3: The social attribute classification component 312 is configured to observe behavioral traits using Artificial Intelligence (AI) to map out a behavioral type, and to personalize human interactive experience based on their behavioral type. See also Crisfalusi at ¶ [0048]: Engagement with a shop assistant to obtain directions to a particular product may indicate a different intention to obtain the product than interactions with accompanying persons, such as family members of friends. It may be easy to distinguish shop assistants from family members if shop assistants wear a uniform or if other persons are wearing clothing which allows them to be readily distinguished from persons accompanying the subject. See also Crisfalusi at ¶ [0048]: The action detection component 308, and the activity detection component 310, with one or more object recognition algorithms designed to distinguish between the different objects that may be held in a subject's hands. For example, distinguishing between a sales product, and the user's mobile phone. With this distinction, the time spent by the subject with items other than retail products in their hands is excluded. See also Crisfalusi at ¶ [0058]: The real-time labelling of social attribute class further assists the retailer with real time interpretation of the effect of changing a customer's local environment from an individual customer perspective or a collective customer group-wise perspective. The real-time labelling of social attribute class further facilitates the retailer with rapid optimization of the customer's environment to maximize the sales outcome. See also Crisfalusi at Fig. 5.)
- associating the certain group with behaviors of the customers who belong to the group (see at least Crisfalusi: Fig. 7 & ¶ [0032-0034] & ¶ [0069-0071]. Crisfalusi notes that the pose detection component may include convolutional neural networks (CNN) and dedicated logic elements for assessing the movements of the subject's joints over time, and detecting patterns associated with specific poses and tracking the subject's pose movements over time, for example, for distinguishing between right and left hands, and linking the joints according to a skeletal frame or sequences of joint movements associated with the adoption of specific poses. See also Crisfalusi at ¶ [0034]: The activity detection component 310 is configured to detect the presence of an observed sequence of actions determined by the action detection component 308 associated with the performance of a pre-defined task. In an example, item selection activity may include multiple actions such as picking up an item, moving item to shopping basket/trolley, and putting the item in the shopping basket/trolley. See also Crisfalusi at ¶ [0039-0040]: The process of aggregating may include evaluating particular actions separately from others, for example, weighting joint speeds associated with walking differently from hand movements in moving items on shelves. See also Crisfalusi at ¶ [0069-0071]: The activity may be detected by detecting the presence of an observed sequence of actions associated with the performance of a pre-defined task. In an example, item selection activity may include multiple actions such as picking up an item, moving item to shopping basket/trolley, and putting the item in the shopping basket/trolley.);
- associating the certain group with the information on the purchase of the product (see at least Crisfalusi: ¶ [0022] & ¶ [0058] & ¶ [0075]. Crisfalusi notes that the real-time labelling of social attribute class further assists the retailer with real time interpretation of the effect of changing a customer's local environment from an individual customer perspective or a collective customer group-wise perspective. The real-time labelling of social attribute class further facilitates the retailer with rapid optimization of the customer's environment to maximize the sales outcome. See also Crisfalusi at ¶ [0075]: At 716, a social attribute class is automatically assigned to the human subject based on the values of the first, second and third social attributes. A social attribute class is assigned to the human subject from a group consisting of ‘driver,’ ‘analytical,’ ‘amiable’ and ‘expressive’. A set of class scores is computed for the human subject, wherein the set includes first, second and third class scores for first, second and third social attributes. See also Crisfalusi at Figs. 2-5.).
It would have been obvious for one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the teachings of Sharma / Li information processing apparatus for customer service purchasing behavior between the sales clerk and customer with the aforementioned teachings of: identifying behaviors of a plurality of customers who belong to the certain group in the store by analyzing the video & identifying a first behavior type that is led by a behavior of each of the customers belonging to the certain group among a plurality of behavior types that define transition of processes of the behaviors that are performed since entrance into the store until purchase of a product in the store & generating information on the purchase of the product by using the first behavior type & associating the certain group with behaviors of the customers who belong to the group & associating the certain group with the information on the purchase of the product, and in further view of Crisfalusi, whereby the real-time labelling of social attribute class enables the retailer to obtain an evolving insight into the customer's social attribute classification and the effect of the customer's interactions with the other related or unrelated individuals in the same field of view. The real-time labelling of social attribute class further assists the retailer with real time interpretation of the effect of changing a customer's local environment from an individual customer perspective or a collective customer group-wise perspective. The real-time labelling of social attribute class further facilitates the retailer with rapid optimization of the customer's environment to maximize the sales outcome (see at least Crisfalusi: ¶ [0058].)
Further, the claimed invention is merely a combination of old elements in a similar field for customer service purchasing behavior between the sales clerk and customer, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that, given the existing technical ability to combine the elements as evidenced by Crisfalusi, the results of the combination were predictable.
Moreover, Sharma / Li / Crisfalusi information processing apparatus for customer service purchasing behavior between the sales clerk and customer does not explicitly disclose, but Guack in the analogous art for customer service purchasing behavior between the sales clerk and customer teaches the following limitations:
- inputting (see at least Guack: ¶ [0078]: “The input interface 120 may include a camera for inputting a video signal, a microphone for receiving an audio signal, and a user input interface for receiving information from a user.”) a first partial image of a first person who is extracted from the video into a machine learning model (see at least Guack: ¶ [0219] & ¶ [0245] & ¶ [0266-0267]. Guack notes that “the tracker can handle partial occlusion or complete occlusions for certain time periods, as described in further detail below in FIG. 12.” See also Guack at ¶ [0245]: The monocular cameras 1303 a, 1303 b manage partial and complete occlusions of users and shelves better than a single depth camera. See also Guack at ¶ [0266-0267]: The output layer 1426 is a task-dependent layer that performs a “computer vision related task such as feature extraction, object recognition, object detection, pose estimation, or the like. Neural networks, such as CNNs are often used to solve computer vision problems including feature extraction, object recognition, object detection, and pose estimation.” See also Guack at ¶ [0310]) that is generated using a partial image of a person extracted from the video as a feature value (see at least Guack: ¶ [0079-0080] & ¶ [0245]. Guack notes that “The learned artificial neural network may be referred to as a learning model. The learning model may be used to an infer result value for new input data rather than learning data, and the inferred value may be used as a basis for determination to perform a certain operation.” The processor 180 or the learning processor 130 may extract an input feature by preprocessing the input data. See also Guack at ¶ [0245]: Guack notes that “The monocular cameras 1303 a, 1303 b manage partial and complete occlusions of users and shelves better than a single depth camera.) and adopting one of the sales clerk and the customer as a correct answer label (see at least Guack: ¶ [0057] & ¶ [0112] & ¶ [0310-0311]. Guack teaches that “the supervised learning may refer to a method of learning an artificial neural network in a state in which a label for learning data is given, and the label may mean the correct answer (or result value) that the artificial neural network must infer when the learning data is input to the artificial neural network. The unsupervised learning may refer to a method of learning an artificial neural network in a state in which a label for learning data is not given. Supervised learning may refer to a method for training an artificial neural network with training data that has been given a label. In addition, the label may refer to a target answer (or a result value) to be guessed by the artificial neural network when the training data is inputted to the artificial neural network. See also Guack at ¶ [0310-0311]: “The association method 1700 b is performed as an initial step when a clerk is stocking a bin on a shelf unit with items. The association method 1700 b provides an initial set of clustering by providing the machine learning model with specific unit weights associated with specific items identified by SKU numbers. For example, the method in block 1701 b may be performed by a store clerk when he is initially stocking a shelf unit with items. The store clerk has a mapping of items, a mapping of the items and their SKU numbers, and a corresponding bin on a shelf unit that the item should be placed in. See also Guack at ¶ [0317]: The training method 1700 c is performed by a store clerk when he is initially stocking a shelf unit (e.g., shelf unit 503 shown in FIGS. 5-7) with items in their corresponding bins on the shelf unit. The training method 1700 c is performed by a store clerk when the store clerk is moving an item from one bin on the shelf unit to another bin on the shelf unit. See also Guack at ¶ [0389]: The module fusion approach finds correspondence between two or more modules, assigns unique ID to people and objects, and corrects the detections.) into a neural network (see at least Guack: ¶ [0052-0057] & ¶ [0201].)
- determine whether the first person is one of the sales clerk and the plurality of customers (see at least Guack: ¶ [0192-0194] & ¶ [0317] & Figs. 6-7. Guack teaches that a shopper 501 (e.g., a first user), a shelf unit 503, an item 505, a retail store employee 507 (e.g., a second user), a retailer application platform 509, and at least one camera 511 (e.g., image sensor). The operating environment 500 is configured to provide shoppers 501 (or users) with an automated shopping experience where shoppers can purchase items without having to go through a self-check-out line, interact with retail store employees 507, or interact with a cashier. See also Guack at [0317]: Training method 1700 c is performed by a store clerk when he is initially stocking a shelf unit (e.g., shelf unit 503 shown in FIGS. 5-7) with items in their corresponding bins on the shelf unit. The training method 1700 c is performed by a store clerk when the store clerk is moving an item from one bin on the shelf unit to another bin on the shelf unit. See also Guack at ¶ [0310-0311].)
It would have been obvious for one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the teachings of Sharma / Li / Crisfalusi information processing apparatus for customer service purchasing behavior between the sales clerk and customer with the aforementioned teachings of: inputting a first partial image of a first person who is extracted from the video into a ML model that is generated through ML using a partial image of a person extracted from the video as a feature value and adopting one of the sales clerk and the customer as a correct answer label into a neural network and determine whether the first person is one of the sales clerk and the plurality of customers, and in further view of Guack, wherein the supervised learning may refer to a method of learning an artificial neural network in a state in which a label for learning data is given, and the label may mean the correct answer (or result value) that the artificial neural network must infer when the learning data is input to the artificial neural network (see at least Guack: ¶ [0057]).
Further, the claimed invention is merely a combination of old elements in a similar field for customer service purchasing behavior between the sales clerk and customer, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that, given the existing technical ability to combine the elements as evidenced by Guack, the results of the combination were predictable.
Regarding Dependent Claims 4, 11 and 17, Sharma / Li / Crisfalusi / Guack non-transitory computer-readable recording medium / method / apparatus for customer service purchasing behavior between the sales clerk and customer teaches the limitations of Independent Claims 1, 8 and 14 above, and Sharma further teaches the non-transitory computer-readable recording medium / method / apparatus for customer service purchasing behavior between the sales clerk and customer comprising:
- wherein the identifying the relationships includes identifying relationships between the sales clerk and the plurality of customers (see at least Sharma: Figs. 7-10 & Col. 8, Lns. 32-39. Sharma teaches that the employees' influence on the customers' activities in the retail space can also be measured by analyzing the relationship between the employee data and the customer behavior analysis in the retail space. The solution leverages the strengths of the technologies in the present invention and processes to deliver a new level of access to the understanding of customer activities, not to mention the employees' activities. Sharma also teaches at Col. 11, Lns. 36-41: “Rules for utilizing the relationship among the tracks of people, in which the relationship shows the employee and non-employee interaction. Examples of such relationship can comprise one-to-many mapping of tracking interaction over a period of time, in which one track interacts with multiple tracks.” Sharma also teaches at Col. 13, Lns. 28-37: Sharma notes captures a plurality of input images, including “video input images 1331 through “video input images N’ 333, of the persons in the physical space 130 by a plurality of means for capturing images 100. The plurality of input images for person detection 710 and person tracking 714. The person detection 710 and person tracking 714 are performed for each person in the plurality of persons in each field of view of the plurality of means for capturing images 100.). by inputting the video (see at least Sharma: Figs. 7-8. Sharma at Fig. 7 teaches video input images 1 331 and video input images N 333.) into the machine learning model if it is determined that the video includes the sales clerk and the plurality of customers (see at least Sharma: Figs. 1-2 & Col. 15, Lns. 19-29 & Figs. 7-8. Sharma teaches that based on the observation on the people trajectories, multiple high-level behavioral classes are defined, such as Stocking, helping customers, or entering/leaving employee area. Then, the system can train a learning machine such as Support Vector Machines, using a number of examples of the trajectories. The whole set or a subset of aforementioned behavioral features are utilized as an input to this machine learning approach. See also Sharma at Col. 4, Lns. 60-67 & Col. 5, Lns. 1-3. Sharma teaches that the learning algorithm-based behavior analysis, in which the features of the behavior comprise duration of dwell time, start, end, number of stops, location of stops, average length of stops, velocity change, repetition of location, displacement of stops, non-uniform pattern, percentage of time spent in a specific employee region, color signature of tracks of the plurality of persons. See also Sharma at Fig. 7 teaches video input images 1 331 and video input images N 333).
- determining whether the relationships between the sales clerk and the plurality of customers indicate a customer service behavior (see at least Sharma: Figs. 7-10 & Col. 8, Lns. 32-36. Sharma notes that the employees' influence on the customers' activities in the retail space can also be measured by analyzing the relationship between the employee data and the customer behavior analysis in the retail space. See also Sharma at Col. 5, Lns. 43-48: The benefits of the automated behavior analysis and employee recognition technology comprise: improvement in physical space performance based on a deep understanding of customer-only behaviors, improvement in the performance and productivity of employees through employee behavior statistics, increased customer service performance through refined employees' interaction with customers, and actual measurement for the correlation between the employees' performance and sales. See also Sharma at Col. 9, Lns. 10-15: Employees' interaction with customers can be refined, which also essentially increases the customer service performance. See also Sharma at Col. 11, Lns. 35-41: Rules for utilizing the relationship among the tracks of people, in which the relationship shows the employee and non-employee interaction.).
Regarding Dependent Claims 6, 13 and 19, Sharma / Li / Crisfalusi / Guack non-transitory computer-readable recording medium / method / apparatus for customer service purchasing behavior between the sales clerk and customer teaches the limitations of Independent Claims 1, 8 and 14 above, and Li further teaches the non-transitory computer-readable recording medium / method / apparatus for customer service purchasing behavior between the sales clerk and customer comprising:
- the classifying into the certain group (see at least Li: Fig. 7 & ¶ [0016-0018] & ¶ [0083-0084]. Li teaches classifying customers into a plurality of customer clusters based on the identifier of the customer, and determining an identifier of a corresponding category based on per-category purchase data regarding each of the plurality of customer clusters.) includes:
- determining that the plurality of customers belong to the different groups (see at least Li: Fig. 7 & ¶ [0023] & ¶ [0162]. Li notes that the server 1310 estimates a position of the customer in the physical store and transmits item information related to the recommended item to an additional display 1320 within a threshold distance from the estimated position of the customer. The providing may include transmitting item information related to the recommended item to a display disposed within a threshold distance from a position of the customer. See also Li at ¶ [0074].) when the group similarity having the penalty value added thereto is equal to or larger than a predefined threshold (see at least Li: ¶ [0139] & ¶ [0162].)
It would have been obvious for one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the teachings of Sharma / Li / Crisfalusi / Guack non-transitory computer-readable recording medium / method / apparatus for customer service purchasing behavior between the sales clerk and customer with the aforementioned teachings of: the classifying into the certain group includes: determining that the plurality of customers belong to the different groups, when the group similarity having the penalty value added thereto is equal to or larger than a predefined threshold, and in further view of Li, whereby the online procedure includes an operation of acquiring a customer image and an operation of automatically determining the purchase tendency model most suitable for the corresponding customer from the purchase tendency model database based on a visual feature of a customer appearance identified from the customer image (see at least Li: ¶ [0068]). Additionally, the customer item recommending apparatus calculates a similarity level between the customers based on an inner product between the customer feature vectors of the individual customers. The customer item recommending apparatus classifies the customers into individual customer clusters based on the similarity level between the customers. The individual customer clusters are clusters each including customers having similar purchase histories (see at least Li: ¶ [0083].)
Further, the claimed invention is merely a combination of old elements in a similar field for customer service purchasing behavior between the sales clerk and customer, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that, given the existing technical ability to combine the elements as evidenced by Li, the results of the combination were predictable.
Regarding Dependent Claim 7, Sharma / Li / Crisfalusi / Guack non-transitory computer-readable recording medium for customer service purchasing behavior between the sales clerk and customer teaches the limitations of Independent Claim 1 above, and Guack further teaches the non-transitory computer-readable recording medium for customer service purchasing behavior between the sales clerk and customer comprising:
- identifying an image area of each of the plurality of customers from the video (see at least Guack: Fig. 6 & ¶ [0078] & ¶ [0372]. Guack notes that the input interface 120 may include a camera for inputting a video signal, a microphone for receiving an audio signal, and a user input interface for receiving information from a user. See also Guack at ¶ [0372]: Guack teaches sending events (such as a customer picking up an item or returning an item) to the cloud network 2005. Each edge device 2007, 2009, 2011, and 2013 is independently connected to and identified on the cloud network or servers 2005. An edge device is responsible for detecting each customers' events (or actions) including picking-up an item, dropping an item, and sending the corresponding events to the cloud servers 2005. See also Guack at Fig. 7 & Figs. 11-12.);
- identifying a position of a skeleton in a person relative to the product (see at least Guack: Fig. 6 & ¶ [0231-0232]. Guack teaches that the processed frame is also processed for human detection to identify the body of the user as different users (step 909) using at least 17 joints in a user's body and an AI model to create a skeletal structure (see skeletal structure 1000 in FIG. 10 and the skeletal structure 1100 in FIG. 11). The skeletal structure is processed for body pose-estimation (step 911) to estimate and track the 3D position of the joints over time using deep learning models to create a first bounding box of the identified user. See also Guack at Fig. 10 and Fig. 11, where at “Fig. 11 of Guack shows a skeleton of a person relative to products on a shelf.”) by inputting the image area of each of the plurality of customers into the neural network (see at least Guack: Figs. 5-7 & ¶ [0201] & ¶ [0372]. Guack notes that the detected skeleton in a 2D image defines the region of the body of the user 501. A deep neural network (described in more detail in FIG. 14) describes the image crop surrounding the body as a feature vector. The vector implicitly describes the body shape, clothing, facial features, or the like of the user 501 for identification over time and across different shelf units 503. As shown in FIG. 7 below, the device generates a detected skeleton (e.g., the skeletal structure 1100 in FIG. 10) in 2D that defines at least 17 regions or joints of a body of the user 501 captured by the camera. See also Guack at Figs. 5 and 7. Guack at ¶ [0189]: “Guack teaches monitoring a customer's shopping behavior for customers to purchase products without being checked out by a cashier or using a self-checkout station. However, such a system can fail to distinguish between items that look similar, smaller items, or items that are occluded by other objects or by a customer's hands.”)
- identifying the behaviors of the plurality of customers based on the position of the skeleton in the person relative to the product (see at least Guack: Fig. 6 & ¶ [0189] & ¶ [0195]. Guack teaches that a computer vision algorithm detects items 505 that are being picked up from the shelf unit 503 or returned back to the shelf unit 503 to add or remove items from a virtual shopping basket for anti-fraud purposes, real-time inventory management, and shopper behavior analytics. A massive network of cameras installed on ceilings for tracking and monitoring a customer's shopping behavior for customers to purchase products without being checked out by a cashier or using a self-checkout station. However, such a system can fail to distinguish between items that look similar, smaller items, or items that are occluded by other objects or by a customer's hands. See also Guack at Fig. 7 & Figs. 11-12.).
It would have been obvious for one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the teachings of Sharma / Li / Crisfalusi / Guack non-transitory computer-readable recording medium for customer service purchasing behavior between the sales clerk and customer with the aforementioned teachings of: identifying an image area of each of plurality of customers from the video & identifying a position of a skeleton in a person relative to the product by inputting the image area of each of plurality of customers into the neural network & identifying the behaviors of a plurality of customers based on the position of a skeleton in a person relative to the product, and in further view of Guack, wherein the supervised learning may refer to a method of learning an artificial neural network in a state in which a label for learning data is given, and the label may mean the correct answer (or result value) that the artificial neural network must infer when the learning data is input to the artificial neural network (see at least Guack: ¶ [0057]). Also, that retail stores have used a combination of computer vision, sensor fusion and deep machine learning to implement these automated shopping experiences (see at least Guack: ¶ [0184]).
Further, the claimed invention is merely a combination of old elements in a similar field for customer service purchasing behavior between the sales clerk and customer, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that, given the existing technical ability to combine the elements as evidenced by Guack, the results of the combination were predictable.
Regarding Dependent Claims 9 and 15, Sharma / Li / Crisfalusi / Guack method / apparatus for customer service purchasing behavior between the sales clerk and customer teaches the limitations of Independent Claims 8 and 14 above, and Crisfalusi further teaches the method / apparatus for customer service purchasing behavior between the sales clerk and customer comprising:
- identifying behaviors of a plurality of customers who belong to the certain group in the store by analyzing the video (see at least Crisfalusi: Fig. 2 & ¶ [0021] & ¶ [0058] & ¶ [0075]. Crisfalusi notes that the environment 100 includes a retail store area 102, an image capturing component 104 for capturing image and video data pertaining to the retail store area 102 in real time, and a central control unit 106 communicatively coupled to various computing devices present in the retail store area 102, and to the image capturing component 104 for classifying and studying the behavior of the customers in the retail store area 102. See also Crisfalusi at FIG. 2 which notes the social behavioral space 200 that illustrates four different types of social attribute classes amongst humans, depending on the distribution of social behaviors within the social behavioral space 200. See also Crisfalusi at ¶ [0058]: The real-time labelling of social attribute class further assists the retailer with real time interpretation of the effect of changing a customer's local environment from an individual customer perspective or a collective customer group-wise perspective. The real-time labelling of social attribute class further facilitates the retailer with rapid optimization of the customer's environment to maximize the sales outcome. See also Crisfalusi at ¶ [0075]: A social attribute class is assigned to the human subject from a group consisting of ‘driver,’ ‘analytical,’ ‘amiable’ and ‘expressive’. In an embodiment of the present disclosure, a set of class scores is computed for the human subject, wherein the set includes first, second and third class scores for first, second and third social attributes. See also Crisfalusi at Fig. 7.);
- identifying a first behavior type that is led by a behavior of each of the customers belonging to the certain group among a plurality of behavior types that define transition of processes of the behaviors (see at least Crisfalusi: Figs. 2-3 & ¶ [0032] & ¶ [0036]. Crisfalusi notes that the pose detection component 306 may include convolutional neural networks (CNN) and dedicated logic elements for assessing the movements of the subject's joints over time, and detecting patterns associated with specific poses and tracking the subject's pose movements over time, for example, for distinguishing between right and left hands, and linking the joints according to a skeletal frame or sequences of joint movements associated with the adoption of specific poses. See also Crisfalusi at ¶ [0027-0028] & Fig. 2. Crisfalusi teaches that the first social attribute class 202 may be referred to as ‘Driver’, according to which a person is extroverted and task focused; and is characterized as being objective, focused, decisive, high energy and fast paced. The second social attribute class 204 may be referred to as ‘Analytical’ according to which a person is introverted and task focused; and is characterized as being fact and task oriented, logical, requiring evidence such as benchmarks and statistics, and needing time to decide. The third social attribute class 206 may be referred to as ‘Amiable’ according to which a person is introverted and people focused; and is characterized as consistent, dependable and stabilizing, cooperative and easy to get along with, indecisive and risk-averse. The fourth social attribute class 208 may be referred to as ‘Expressive’ according to which a person is extroverted and people focused; and is characterized as being relationship-oriented, verbally and socially adept, status conscious and a big-picture thinker. See also Crisfalusi at ¶ [0036]: “The social attribute classification component 312 is configured to observe behavioral traits using Artificial Intelligence (AI) to map out a behavioral type, and to personalize human interactive experience based on their behavioral type”.) that are performed since entrance into the store until purchase of a product in the store (see at least Crisfalusi: Fig. 5 & ¶ [0037] & ¶ [0062-0064]. Crisfalusi teaches that the interaction time attribute (Tint) is the average time spent by the customers 108 interacting with people or responding to a trigger, for example, in-store displayed adverts or alternative forced routes in the store or people welcoming customers at the entrance to the store; and the analysis time attribute (Tanal) includes the average time spent by the customers 108 in analyzing a product. See also Crisfalusi at ¶ [0024]: The scanning zone is an area in front of the scanner where the user brings up the items for scanning for the purpose of buying of those items. See also Crisfalusi at ¶ [0062]: The system 506 may assign a unique ID to each customer upon their entry in the retail store area 502 that may be maintained over the entire duration of the customer's stay in the retail store area 502, for example, during the customer's entire journey from the entrance of the store to exit from targeted area(s) of the store. The social attribute classes assigned to a customer may be computed at any given time using the entire history of attribute values observed up to that point in the customer's journey. If there are specific places with advertising panels in the store, the customer's social attribute classification may be determined every time the customer reaches a specific location in front of such panel. See also Crisfalusi at ¶ [0062]: The system 506 may be configured to maintain consistent labelling of the customer during their stay in the store by detecting when a given customer enters the field of view of proximal cameras, using crude pattern recognition identification approach, and face recognition approach. The system 506 may assign a unique ID to each customer upon their entry in the retail store area 502 that may be maintained over the entire duration of the customer's stay in the retail store area 502, for example, during the customer's entire journey from the entrance of the store to exit from targeted area(s) of the store. The social attribute classes assigned to a customer may be computed at any given time using the entire history of attribute values observed up to that point in the customer's journey.)
- generating information on the purchase of the product by using the first behavior type (see at least Crisfalusi: ¶ [0022] & ¶ [0040-0048] & ¶ [0058]. Crisfalusi notes that the retail store area 102 includes first through sixth customers 108 a, 108 b, 108 c, 108 d, 108 e, and 108 f that are configured to store items for purchase by the customers 108, to enable the customers 108 to bill their products. See also Crisfalusi at ¶ [0036] & Fig. 3: The social attribute classification component 312 is configured to observe behavioral traits using Artificial Intelligence (AI) to map out a behavioral type, and to personalize human interactive experience based on their behavioral type. See also Crisfalusi at ¶ [0048]: Engagement with a shop assistant to obtain directions to a particular product may indicate a different intention to obtain the product than interactions with accompanying persons, such as family members of friends. It may be easy to distinguish shop assistants from family members if shop assistants wear a uniform or if other persons are wearing clothing which allows them to be readily distinguished from persons accompanying the subject. See also Crisfalusi at ¶ [0048]: The action detection component 308, and the activity detection component 310, with one or more object recognition algorithms designed to distinguish between the different objects that may be held in a subject's hands. For example, distinguishing between a sales product, and the user's mobile phone. With this distinction, the time spent by the subject with items other than retail products in their hands is excluded. See also Crisfalusi at ¶ [0058]: The real-time labelling of social attribute class further assists the retailer with real time interpretation of the effect of changing a customer's local environment from an individual customer perspective or a collective customer group-wise perspective. The real-time labelling of social attribute class further facilitates the retailer with rapid optimization of the customer's environment to maximize the sales outcome. See also Crisfalusi at Fig. 5.)
- associating the certain group with the information on the purchase of the product (see at least Crisfalusi: ¶ [0022] & ¶ [0058] & ¶ [0075]. Crisfalusi notes that the real-time labelling of social attribute class further assists the retailer with real time interpretation of the effect of changing a customer's local environment from an individual customer perspective or a collective customer group-wise perspective. The real-time labelling of social attribute class further facilitates the retailer with rapid optimization of the customer's environment to maximize the sales outcome. See also Crisfalusi at ¶ [0075]: At 716, a social attribute class is automatically assigned to the human subject based on the values of the first, second and third social attributes. A social attribute class is assigned to the human subject from a group consisting of ‘driver,’ ‘analytical,’ ‘amiable’ and ‘expressive’. A set of class scores is computed for the human subject, wherein the set includes first, second and third class scores for first, second and third social attributes. See also Crisfalusi at Figs. 2-5.).
It would have been obvious for one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the teachings of Sharma / Li / Crisfalusi / Guack method / apparatus for customer service purchasing behavior between the sales clerk and customer with the aforementioned teachings of: identifying behaviors of a plurality of customers who belong to the certain group in the store by analyzing the video & identifying a first behavior type that is led by a behavior of each of the customers belonging to the certain group among a plurality of behavior types that define transition of processes of the behaviors that are performed since entrance into the store until purchase of a product in the store & generating information on the purchase of the product by using the first behavior type & associating the certain group with the information on the purchase of the product, and in further view of Crisfalusi, whereby the real-time labelling of social attribute class enables the retailer to obtain an evolving insight into the customer's social attribute classification and the effect of the customer's interactions with the other related or unrelated individuals in the same field of view. The real-time labelling of social attribute class further assists the retailer with real time interpretation of the effect of changing a customer's local environment from an individual customer perspective or a collective customer group-wise perspective. The real-time labelling of social attribute class further facilitates the retailer with rapid optimization of the customer's environment to maximize the sales outcome (see at least Crisfalusi: ¶ [0058].)
Further, the claimed invention is merely a combination of old elements in a similar field for customer service purchasing behavior between the sales clerk and customer, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that, given the existing technical ability to combine the elements as evidenced by Crisfalusi, the results of the combination were predictable.
Regarding Dependent Claim 21, Sharma / Li / Crisfalusi / Guack non-transitory computer-readable recording medium for customer service purchasing behavior between the sales clerk and customer teaches the limitations of Claims 1 and 7 above, and Guack further teaches the non-transitory computer-readable recording medium for customer service purchasing behavior between the sales clerk and customer comprising:
- wherein identifying the behaviors (see at least Guack: Fig. 8 & ¶ [0211].) includes calculating an angle of an elbow of the skeleton based on a first vector between the elbow and a shoulder and a second vector between the elbow and a wrist (see at least Guack: ¶ [0215-0216] & ¶ [0220-0224] & ¶ [0236]. Guack teaches that FIG. 10 shows that the at least 17 joints of a user may include (but is not limited to): a head, a neck, a left and right collar, a left shoulder, a right shoulder, a left elbow, a right elbow, a torso, a waist, a left wrist, a right wrist, a left hand, a right hand, a left hip, a right hip. See also Guack at Figs. 10-11.), and determining that the person is extending a hand to the product when the calculated angle exceeds a predetermined threshold (see at least Guack: ¶ [0305] & ¶ [0319] & ¶ [0329] & Fig. 12. Guack at Fig. 12 step 1201 showing original image with hand detection of extending a hand to touch a product/item.).
It would have been obvious for one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the teachings of Sharma / Li / Crisfalusi / Guack non-transitory computer-readable recording medium for customer service purchasing behavior between the sales clerk and customer with the aforementioned teachings of: wherein identifying the behaviors includes calculating an angle of an elbow of the skeleton based on a first vector between the elbow and a shoulder and a second vector between the elbow and a wrist, and determining that the person is extending a hand to the product when the calculated angle exceeds a predetermined threshold, and in further view of Guack, wherein the supervised learning may refer to a method of learning an artificial neural network in a state in which a label for learning data is given, and the label may mean the correct answer (or result value) that the artificial neural network must infer when the learning data is input to the artificial neural network (see at least Guack: ¶ [0057]). Also, that retail stores have used a combination of computer vision, sensor fusion and deep machine learning to implement these automated shopping experiences (see at least Guack: ¶ [0184]).
Further, the claimed invention is merely a combination of old elements in a similar field for customer service purchasing behavior between the sales clerk and customer, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that, given the existing technical ability to combine the elements as evidenced by Guack, the results of the combination were predictable.
Regarding Dependent Claim 22, Sharma / Li / Crisfalusi / Guack non-transitory computer-readable recording medium for customer service purchasing behavior between the sales clerk and customer teaches the limitations of Claims 1 and 7 above, and Crisfalusi further teaches the non-transitory computer-readable recording medium for customer service purchasing behavior between the sales clerk and customer comprising:
- wherein identifying the behaviors includes tracking a transition of the position of the skeleton across a plurality of consecutive frames of the video (see at least Crisfalusi: Fig. 7 & ¶ [0032] & ¶ [0069]. See also Crisfalusi at Fig. 7 step 704 noting “At step 704, pose detection and tracking of a human subject in real-time is performed by analyzing the image and video data. Also detecting patterns associated with specific poses and tracking the subject's pose movements over time, for example, for distinguishing between right and left hands, and linking the joints according to a skeletal frame or sequences of joint movements associated with the adoption of specific poses.”), and recognizing a motion of the person relative to the product (see at least Crisfalusi: ¶ [0032] & ¶ [0048]. Crisfalusi notes distinguishing between a sales product, and the user's mobile phone. With this distinction, the time spent by the subject with items other than retail products in their hands is excluded.) based on a change in the position of the skeleton exceeding a predetermined value during the plurality of consecutive frames (see at least Crisfalusi: ¶ [0032] & ¶ [0039] & ¶ [0054]. Crisfalusi notes that the image capturing component 302 may include multiple overhead cameras disposed at different angles, so to enable the pose detection component 306 to analyze specific aspects of the subject's skeletal anatomy, and perform motion estimation and movement prediction of different body part junctions. See also Crisfalusi at ¶ [0069].).
It would have been obvious for one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the teachings of Sharma / Li / Crisfalusi / Guack non-transitory computer-readable recording medium for customer service purchasing behavior between the sales clerk and customer with the aforementioned teachings of: wherein identifying the behaviors includes tracking a transition of the position of the skeleton across a plurality of consecutive frames of the video, and recognizing a motion of the person relative to the product based on a change in the position of the skeleton exceeding a predetermined value during the plurality of consecutive frames, and in further view of Crisfalusi, whereby the real-time labelling of social attribute class enables the retailer to obtain an evolving insight into the customer's social attribute classification and the effect of the customer's interactions with the other related or unrelated individuals in the same field of view. The real-time labelling of social attribute class further assists the retailer with real time interpretation of the effect of changing a customer's local environment from an individual customer perspective or a collective customer group-wise perspective. The real-time labelling of social attribute class further facilitates the retailer with rapid optimization of the customer's environment to maximize the sales outcome (see at least Crisfalusi: ¶ [0058].)
Further, the claimed invention is merely a combination of old elements in a similar field for customer service purchasing behavior between the sales clerk and customer, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that, given the existing technical ability to combine the elements as evidenced by Crisfalusi, the results of the combination were predictable.
17. Claims 5, 12 and 18 is rejected under 35 U.S.C. 103 as being unpatentable over Sharma / Li / Crisfalusi / Guack, and in further view of Foreign Patent Application (JP 2017191495A) hereinafter Masuda, et. al.
Regarding Dependent Claims 5, 12 and 18, Sharma / Li / Crisfalusi / Guack non-transitory computer-readable recording medium for customer service purchasing behavior between the sales clerk and customer does not explicitly disclose, but hereinafter Masuda in the analogous art for customer service purchasing behavior between the sales clerk and customer teaches the following limitations:
- generating a customer service history of the sales clerk on the basis of the plurality of customers who are identified as having received the customer services from the sales clerk (see at least Masuda: 4th-6th ¶’s of Page 5 & 3rd ¶ of Page 9. Masuda teaches that the customer service history may include information on the store clerk who served the customer, the contents of the service, and the like. The purchase history may include purchased products, number, amount, and the like. Further, the additional data may include matters and features that should be stored by the store clerk. It is preferable that at least a part of the attribute data is automatically updated. A part of attribute data may be registered by a store clerk or the like. See also Masuda at 3rd ¶ of Page 9: Masuda notes that the apparatus 100 may include an update unit that updates attribute data corresponding to terminal identification information in response to a store clerk operating a terminal in the store in response to purchase of a product of a holder, use of a service, and the like. . In this case, as an example, the update unit includes a reception unit that receives information input by the store clerk operating the terminal. Further, when the store clerk responds to a new customer, the update unit may newly register the store clerk as a responder according to the operation of the terminal.);
- the classifying into the certain group includes, if the sales clerk provided the customer services to each of the plurality of customers on the basis of the customer service history (see at least Masuda: 4th-6th ¶’s of Page 5 & 3rd ¶ of Page 9.), determining that the plurality of customers belong to the different groups (see at least Masuda: 6th-8th ¶’s of Page 13. Masuda notes that a customer may act as a plurality of customer groups. In this case, the apparatus 100 may transmit the provision information corresponding to each attribute data to the holders of the plurality of portable terminals, but it is preferable to transmit the provision information common to the plurality of holders. When the determination unit 130 determines that a plurality of portable terminals are moving in a group, the determination unit 130 provides the plurality of holders based on the attribute data of the plurality of holders corresponding to the plurality of portable terminals. Moreover, the determination part 130 may determine the provision information according to the preference, purchasing history, etc. which are common to several holders among the attribute data of several holders. Further, as an example, the determination unit 130 may prepare a plurality of coupons having different attribute conditions as coupon information in advance, and determine a matching coupon according to the fact that the holder's attribute matches the coupon condition. See also Masuda at 5th ¶ of Page 14.).
It would have been obvious for one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the teachings of Sharma / Li / Crisfalusi / Guack non-transitory computer-readable recording medium for customer service purchasing behavior between the sales clerk and customer with the aforementioned teachings of: generating a customer service history of the sales clerk on the basis of the plurality of customers who area identified as having received the customer services from the sales clerk and the classifying into the certain groups includes, if the sales clerk provided the customer services to each of the plurality of customers on the basis of the customer service history, determining that the plurality of customers belong to the different groups, and in further view of Masuda, whereby the update unit that updates attribute data corresponding to terminal identification information in response to a store clerk operating a terminal in the store in response to purchase of a product of a holder and use of a service. Further, when the store clerk responds to a new customer, the update unit may newly register the store clerk as a responder according to the operation of the terminal (see at least Masuda: (3rd ¶ of Page 9).)
Further, the claimed invention is merely a combination of old elements in a similar field for customer service purchasing behavior between the sales clerk and customer, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that, given the existing technical ability to combine the elements as evidenced by Masuda, the results of the combination were predictable.
18. Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Sharma / Li / Crisfalusi / Guack, and in further view of US PG Pub (US 2024/0378923 A1) hereinafter Makabe, et. al.
Regarding Dependent Claim 20, Sharma / Li / Crisfalusi / Guack non-transitory computer-readable recording medium for customer service purchasing behavior between the sales clerk and customer does not explicitly disclose, but hereinafter Makabe in the analogous art for customer service purchasing behavior between the sales clerk and customer teaches the following limitations:
- wherein identifying the behaviors (see at least Makabe: ¶ [0082-0083] & ¶ [0087]. The interest information acquisition unit 20 acquires interest information on the basis of behavior of the user on the arrival floor. Here, the behavior of the user includes information such as, for example, a period during which the user stays on the arrival floor of the area and a concern direction which is a direction in which the user shows interest of the area on the arrival floor.) includes determining a face orientation of the person based on a vector defined by a midpoint between ears of the skeleton and a nose of the skeleton (see at least Makabe: ¶ [0109-0110] & ¶ [0354]. Makabe notes that the concern direction of the user is represented as a direction from a midpoint of a line segment connecting positions of both shoulders toward a position of the nose. Here, it is only necessary to capture feature amounts of the nose as the feature amounts to be used as the concern direction information regardless of whether or not the nose of the user is covered with a face mask, or the like, that is, whether or not the naked nose itself of the user is in the image or images. Feature amounts of both shoulders and nose obtained using skeleton information of the user. Still further, the concern direction information may be represented using other feature amounts obtained using the skeleton information. See also Makabe at ¶ [0082]: The layout of the user position includes, for example, information on a floor on which the user is located, a coordinate of the user on the floor, orientation of the user. See also Makabe at ¶ [0110]: “The direction indicating interest of the user is an extension in a direction from a midpoint of a line segment connecting positions of both shoulders of the user toward the nose”. See also Makabe at ¶ [0354]: The interest information storage unit 21 stores, for example, identification information unique to the user, interest information of the user and information on a time point and a location at which the interest information is acquired in association with one another. See also Makabe at Figs. 5A-5E showing the vectors.), and identifying that the person is looking at the product based on the face orientation (see at least Makabe: ¶ [0082] & ¶ [0100-0101] & Figs. 5A-5E. See also Makabe at ¶ [0361-0362].)
It would have been obvious for one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the teachings of Sharma / Li / Crisfalusi / Guack non-transitory computer-readable recording medium for customer service purchasing behavior between the sales clerk and customer with the aforementioned teachings of: wherein identifying the behaviors includes determining a face orientation of the person based on a vector defined by a midpoint between ears of the skeleton and a nose of the skeleton, and identifying that the person is looking at the product based on the face orientation, and in further view of Makabe, in order for the user specification unit specifies the users erroneously specified as the same user, as users different from each other. When the user specification unit specifies the users as users different from each other, the user specification unit extracts a difference in a feature amount or feature amounts of the users from the acquired image or images, improves accuracy of specification of the user and resettles specification of the users different from each other (see at least Makabe: ¶ [0085]).
Further, the claimed invention is merely a combination of old elements in a similar field for customer service purchasing behavior between the sales clerk and customer, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that, given the existing technical ability to combine the elements as evidenced by Makabe, the results of the combination were predictable.
19. Claim 23 is rejected under 35 U.S.C. 103 as being unpatentable over Sharma / Li / Crisfalusi / Guack, and in further view of US Patent # (US 10,839,203 B1) hereinafter Guigues, et. al.
Regarding Dependent Claim 23, Sharma / Li / Crisfalusi / Guack non-transitory computer-readable recording medium for customer service purchasing behavior between the sales clerk and customer does not explicitly disclose, but hereinafter Guigues in the analogous art for customer service purchasing behavior between the sales clerk and customer teaches the following limitations:
- wherein identifying the behaviors includes determining a twist of a hip of the person by calculating a rotation angle between a vector connecting a left shoulder and a right shoulder and a vector connecting a left hip and a right hip (see at least Guigues: Fig. 1J & Figs. 9G-9L & Col. 11, Lns. 35-44. Guigues notes that the imaging device may include one or more actuated or motorized features for the angular orientation (e.g., the roll angle, the pitch angle or the yaw angle), by causing a change in the distance between the sensor and the lens, a change in the location of the imaging device, or a change in one or more of the angles defining the angular orientation. See also Guigues at Col. 22, Lns. 28-36: A classifier may be trained to recognize any number of body parts including but not limited to heads, necks, and left or right shoulders, elbows, wrists, hands, hips, knees, ankles, or others. See for example Guigues at Fig. 1J showing “Model Motion of Actor with Each Body Part as Vectors”. See for example Guigues at Fig. 9L showing left hip 960-11-1 and right hip 960-12-1 and left shoulder 960-3-1 and right shoulder 960-4-1. See also Guigues at Col. 6, Lns. 3-35: “Each detection of a body part may include not only a
position of the body part within an image frame, but also a set of vectors extending from the position of the body part to possible positions of other body parts within the image frame”. See also Guigues at Col. 8, Lns. 16-45: “Vectors or trajectories representative of motion of the individual body parts”.).
It would have been obvious for one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the teachings of Sharma / Li / Crisfalusi / Guack non-transitory computer-readable recording medium for customer service purchasing behavior between the sales clerk and customer with the aforementioned teachings of: wherein identifying the behaviors includes determining a twist of a hip of the person by calculating a rotation angle between a vector connecting a left shoulder and a right shoulder and a vector connecting a left hip and a right hip, and in further view of Guigues, whereby images are captured by the imaging devices may be synchronized and provided to a classifier to recognize body parts within the images, and score maps indicative of locations of peak probabilities that the images include the respective body parts may be generated. Locations of peak values within the score maps may be correlated with one another to confirm that a given body part is depicted in two or more fields of view, and vectors indicative of distances to or ranges of motion of body parts, with respect to the given body part, may be generated (see at least Guigues: ¶ [abstract]).
Further, the claimed invention is merely a combination of old elements in a similar field for customer service purchasing behavior between the sales clerk and customer, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that, given the existing technical ability to combine the elements as evidenced by Guigues, the results of the combination were predictable.
20. Claim 24 is rejected under 35 U.S.C. 103 as being unpatentable over Sharma / Li / Crisfalusi / Guack, and in further view of Foreign Patent Application (CN 114761983 A) hereinafter Crisfalusi 2.
Regarding Dependent Claim 24, Sharma / Li / Crisfalusi / Guack non-transitory computer-readable recording medium for customer service purchasing behavior between the sales clerk and customer does not explicitly disclose, but hereinafter Crisfalusi 2 in the analogous art for customer service purchasing behavior between the sales clerk and customer teaches the following limitations:
- wherein identifying relationships (see at least Crisfalusi 2: (1st ¶ of Page 6) & (last ¶ of Page 9). Crisfalusi 2 notes that a fourth social style category 208 may be referred to as "Expressive", according to which a person is outgoing and person-centered; characterized by a focus on relationships, being verbal and social, status-conscious and big-picture thinking. See also Crisfalusi 2: last ¶ of Page 9. Sampling measurements of social style attributes may be performed on a statistically random set of at least 1000-5000 customers to calibrate the first social style attribute s, the second social style attribute T int and the third social style attribute The relationship between Tanal and social style categories.) includes inputting the video into a Human Object Interaction Detection (HOID) model (see at least Crisfalusi 2: Figs. 2-3 & Fig. 5 & (last ¶ of Page 2). Crisfalusi 2 teaches that the system includes an image capture component configured to capture image and video data of a human subject in real time. The system may also include a gesture detection component configured to perform gesture detection and tracking of the human subject in real-time by analyzing the image and video data. See also Crisfalusi 2 at 5th ¶ of Page 7: Social style category component 312 is configured to observe customer behavioral characteristics using visual and other sensory input (e.g., voice from image capture component 302).) trained to output a probability value of interaction between a class indicating a sales clerk and a class indicating a customer (see at least Crisfalusi 2: (4th ¶ of Page 7) & (2nd ¶ of Page 8) & (4th ¶ of Page 19). Crisfalusi 2 teaches that brief casual encounters (e.g., inadvertently meeting each other) have different durations than longer human-to-human conversations. It may also be useful to distinguish who the subject is in contact with. For example, dealing with a store clerk to obtain instructions for a particular product may indicate a different intent to obtain that product than an interaction with an accompanying person (e.g., a friend's family member). It is easy to distinguish a clerk from a family member if the clerk wears a uniform, or if other people wear clothing that allows them to be easily distinguished from those accompanying the subject. See also Crisfalusi 2 at Page 7: The classifiers and logic elements used in pose detection component 306, motion detection component 308, and activity detection component 310 may be trained based on pre-labeled video sequences used in supervised or semi-supervised learning methods. See also Crisfalusi 2 at Page 19: Crisfalusi 2 notes that system 604 is configured to assign social styles to customers based on the routes selected by the customers. The first route x is longer, with no people, and the designed probability of human interaction is very low, while the second route y is shorter, with many people, and the designed human interaction probability is very high. Depending on the route chosen, customers may express a preference that may categorize them in part as having task-oriented or people-oriented characteristics. In addition, the option to select one of the first route x and the second route y may be implemented at an early contact stage with the customer, such as near the entrance of a retail store, to force an early decision, thereby facilitating the basis of the resulting map.)
It would have been obvious for one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the teachings of Sharma / Li / Crisfalusi / Guack non-transitory computer-readable recording medium for customer service purchasing behavior between the sales clerk and customer with the aforementioned teachings of: wherein identifying relationships includes inputting the video into a Human Object Interaction Detection (HOID) model trained to output a probability value of interaction between a class indicating the sales clerk and a class indicating the customer, and in further view of Crisfalusi 2, whereby the activity of the human subject is detected by correlating the detected action sequences. The activity can be detected by detecting the presence of an observed sequence of actions associated with performing a predefined task. An item selection activity may include actions such as picking up an item, moving an item to a basket/cart, and placing an item in the basket/cart. Activity detection may be performed using recurrent neural networks or other classifiers such as a single multi-layer classifier (see at least Crisfalusi 2: 3rd – 5th ¶s of Page 20).
Further, the claimed invention is merely a combination of old elements in a similar field for customer service purchasing behavior between the sales clerk and customer, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that, given the existing technical ability to combine the elements as evidenced by Crisfalusi 2, the results of the combination were predictable.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DERICK HOLZMACHER whose telephone number is (571) 270-7853. The examiner can normally be reached on Monday-Friday 9:00 AM – 6:30 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, Applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Brian Epstein can be reached on 571-270-5389. The fax phone number for the organization where this application or proceeding is assigned is 571-270-8853.
Information regarding the status of an application may be obtained from Patent Center. Status information for published applications may be obtained from Patent Center. Status information for unpublished applications is available through Patent Center for authorized users only. Should you have questions about access to Patent Center, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
/DERICK J HOLZMACHER/Patent Examiner, Art Unit 3625A
/BRIAN M EPSTEIN/Supervisory Patent Examiner, Art Unit 3625