DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 06/25/2026 has been entered.
Status of Claims
Claims 2-11, 13-20, and 22 are pending. Claims 1, 12, and 21 are canceled, and Claim 22 is new.
Response to Arguments
Applicant’s arguments, see p.22-25, filed 06/25/2026, with respect to the rejections of Claim 21 under 35 U.S.C. 101 have been fully considered and are persuasive. The cancellation of Claim 21 has rendered the 35 U.S.C. 101 rejection moot, so the rejection of Claim 21 under this section of the Rules has been withdrawn.
Applicant’s arguments, see p.22-25, filed 06/25/2026, with respect to the rejections of Claims 9 and 21 under 35 U.S.C. 112(b) have been fully considered and are persuasive. The amendments to claim 9 and the cancellation of Claim 21 renders the 35 U.S.C. 112(b) rejection moot. Therefore, the rejection of Claims 9 and 21 under this section of the Rules has been withdrawn.
Applicant’s arguments, see p.22-25, filed 06/25/2026, with respect to the rejections of Claims 2-11, 13-20, and 22 under 35 U.S.C. 103 have been fully considered but are moot because Applicant’s amendments of independent claims has altered the scope of the claims, and therefore, necessitated new grounds of rejection, which are presented below. Examiner has considered applicants arguments with respect to the new claim 22. However, arguments are moot due to new claims being presented and are therefore being analyzed as presented below.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
Use of the word “means” (or “step for”) in a claim with functional language creates a rebuttable presumption that the claim element is to be treated in accordance with 35 U.S.C. 112(f) (pre-AIA 35 U.S.C. 112, sixth paragraph). The presumption that 35 U.S.C. 112(f) (pre-AIA 35 U.S.C. 112, sixth paragraph) is invoked is rebutted when the function is recited with sufficient structure, material, or acts within the claim itself to entirely perform the recited function.
Absence of the word “means” (or “step for”) in a claim creates a rebuttable presumption that the claim element is not to be treated in accordance with 35 U.S.C. 112(f) (pre-AIA 35 U.S.C. 112, sixth paragraph). The presumption that 35 U.S.C. 112(f) (pre-AIA 35 U.S.C. 112, sixth paragraph) is not invoked is rebutted when the claim element recites function but fails to recite sufficiently definite structure, material or acts to perform that function.
Claim elements in this application that use the word “means” (or “step for”) are presumed to invoke 35 U.S.C. 112(f) except as otherwise indicated in an Office action. Similarly, claim elements that do not use the word “means” (or “step for”) are presumed not to invoke 35 U.S.C. 112(f) except as otherwise indicated in an Office action.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are: “a product identifier” and “a customer behavior determinator” relevant to Claims 2-11 and 22.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 2-3, 6-7, 13-14, 17-18, and 22 are rejected under 35 U.S.C. 103 as being unpatentable over Moller et al. (US 20230058903 A1) in view of Johnson et al. (US 20230186266 A1) and Gabale (US 20240169735 A1).
Regarding Claim 2, Moller teaches "A self-checkout apparatus comprising: a pre-identification product table (21); a product recognition table (11); a post-identification product table (31); the product recognition table (11) being provided with a product identification zone (11-1) and on which a product to be identified is located"; (Moller, Fig. 5 and Paras. 31 and 97-100, teaches a self-checkout system comprising a first area, a second area, and a third area wherein a first area may be seen as an unpacking area, a second area which may be seen as a scanning area arranged with an identification means configured to identify the articles wherein the scanning means is in the form of a barcode reader, and a third area may be seen as a packing area where the customer is placing its identified articles , i.e., a self-checkout apparatus comprising a pre-identification product table being the first area packing area as well as a product recognition table with an identification zone for identifying products being the second area with a scanning means and a post-identification product table being the third area packing area);
"
"
"
"a product identifier which detects an identification code assigned to the product from the video of the product obtained by the first camera and interprets the detected identification code to output an identification result of the product captured by the first camera"; (Moller, Fig. 5 and Paras. 40, 42, and 97-99, teaches different sensors being used such as a camera wherein the sensor arrangement comprises an identification means such as a barcode reader wherein the second area has a scanning area arranged with an identification means to identify articles with a scanning means such as a barcode reader wherein information related to the identified article is then displayed, i.e., product identifier to detect an identification code being the barcode assigned to the product from the camera sensor to output the identification result of the product captured by the camera sensor);
"a customer behavior determinator which interprets the video of the surveillance area obtained by the second camera to output a determination result of a customer's behavior captured by the second camera"; (Moller, Paras. 5 and 40, teaches a customer behavioral system comprising a sensor arrangement with one or more sensors and a behavioral analysis module configured to determine at least one behavioral feature of a customer based on sensor data from said one or more sensors to determine behavioral information, i.e., customer behavior determinator which interprets the video of the surveillance area of a second camera sensor to output a determination result of the customer's behavior captured by the second camera);
"a processor which specifies the customer's behavior as an abnormal behavior and a normal behavior based on the identification result of the product captured by the first camera and the determination result of the customer's behavior captured by the second camera, and generates a control signal depending on the specified result"; (Moller, Paras. 85-87, teaches the behavior system using behavioral information to detect if a customer is using the checkout counter the right way wherein the system will not prompt or guide the customer further or if the customer is acting in a deviant way wherein an output means may transmit a signal to the checkout system in the store or transmit instructions to execute an anti-theft operation, i.e., a processor that specifies the customer's behavior as normal being the correct way of operation and abnormal as deviant in which a control signal is generated based on the result);
"and wherein the customer behavior determinator comprises an artificial neural network which analyzes the video of the surveillance area obtained by the second camera to determine the customer's behavior captured by the second camera"; (Moller, Paras. 22 and 45, teaches using convolutional neural networks to process the data gathered from the at least one sensor or plurality of sensors to determine behavioral information, i.e., customer behavior determinator comprises an artificial neural network to analyze the video of the area to determine customer's behavior);
"and wherein the artificial neural network is trained by learning a video of the customer's behavior captured by the second camera as the product is identified by the product identifier and a video of the customer's behavior captured by the second camera as the product is not identified by the product identifier to determine the customer's behavior that does not correspond to the behavior of locating a product in the product identification zone to be identified"; (Moller, Paras. 77-78, 91 and 109, teaches behavioral information being used to determine and output statistical data which is used to learn and update the system by using the statistical data as an input when determining future behavioral information wherein behavioral information includes determining the probability level related to the likelihood a customer will steal based on a failure to scan one or more articles, i.e., neural network is trained using video of customer behavior captured by the camera sensor as the product is identified by the product identifier and using video of customer behavior when the product is not identified by the product identifier to determine customer behavior that does not correspond to the behavior of locating a product in the product identification zone to be identified that is the likelihood of the customer stealing or leaving un-scanned items as the abnormal behavior).
However, Moller does not explicitly teach "a first camera having its photographing area defined by the product identification zone (11-1) of the product recognition table (11); the first camera obtaining a video of the product in the product identification zone (11-1) by photographing the product identification zone (11-1); a second camera which obtains a video of a surveillance area by photographing the surveillance area including the product recognition table (11) and the inside of a store".
In an analogous field of endeavor, Johnson teaches "a first camera having its photographing area defined by the product identification zone (11-1) of the product recognition table (11); (Johnson, FIG. 2A and Paras. 17-18, teaches a self-checkout kiosk which includes a scan area which may include a scanner scale which can weigh and scan items wherein the kiosk also includes a camera that has a field of view defined on the scan area wherein Figure 2A uses dashed lines to define the field of view of the camera as around the scanner scale area used to scan items, i.e., first camera has its photographing area defined by the product identification zone of the recognition table being the camera field of view defined by the scanner scale area of the scan area);
"the first camera obtaining a video of the product in the product identification zone (11-1) by photographing the product identification zone (11-1)"; (Johnson, FIG. 2A and Para. 37, teaches the camera may continuously capture video in the scanning area to detect at least one target object , i.e., first camera obtains video of the product in the identification zone by photographing the identification zone).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Moller by including the camera having a photographing area defined by the identification zone for obtaining video of the product taught by Johnson. One of ordinary skill in the art would be motivated to combine the references since it improves measurement processing (Johnson, Para. 11, teaches the motivation of combination to be to improve techniques for performing measurement information processing).
However, the combination of references of Moller in view of Johnson does not explicitly teach "a second camera which obtains a video of a surveillance area by photographing the surveillance area including the product recognition table (11) and the inside of a store".
In an analogous field of endeavor, Gabale teaches "a second camera which obtains a video of a surveillance area by photographing the surveillance area including the product recognition table (11) and the inside of a store"; (Gabale, Fig. 3 and Para. 6, teaches configuring a camera at the self-checkout counter or on a surface of the physical retail store environment wherein a second camera video stream reidentifies the product object in association with the at least one person object by tracking movement of the product object in the physical retail store environment, i.e., a second camera obtains video of a surveillance area by photographing the surveillance area including the product recognition table and the inside of a store).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Moller and Johnson by including a second camera to obtain video of a surveillance area within the store including the product recognition table taught by Gabale. One of ordinary skill in the art would be motivated to combine the references since it prevents shrinkages in a retail environment (Gabale, Para. 1, teaches the motivation of combination to be to use real-time camera feeds to prevent shrinkages in a retail environment).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date.
Regarding Claim 3, the combination of references of Moller in view of Johnson and Gabale teaches "The self-checkout apparatus of claim 2, wherein the processor analyzes the video of the surveillance area obtained by the second camera to specify the customer's behavior captured by the second camera as an abnormal behavior and a normal behavior based on at least one of a change in facial expression, a change in face color, and a heart rate reading of the customer captured by the second camera, and generates a control signal depending on the specified result"; (Moller, Paras. 53-58 and 82, teaches behavioral features used to determine behavioral information including facial expressions, analysis of facial coloration of the customer, and an approximation of the heart rate of the customer wherein a communication signal may be transmitted based on the determined behavioral information, i.e., determine behavioral information as normal or abnormal based on changes in facial expression, face color, and heart rate of the customer and generate a control signal depending on the result).
Regarding Claim 6, the combination of references of Moller in view of Johnson and Gabale teaches "The self-checkout apparatus of claim 2, further comprising a pre-identification product table and a post-identification product table disposed in a predetermined area around the product recognition table, wherein the abnormal behavior of the customer include: a situation where the customer behavior determinator recognizes two or more products in the customer's hand"; (Moller, Fig. 5 and Paras. 98-100 and 118-120, teaches a first area which is an unpacking area where the customer places his articles that are to be purchased and a third area which is a packing area for the customer to place its identified articles, i.e., pre-identification product table and a post-identification product table disposed in a predetermined area around the product recognition table, and wherein the behavioral analysis module may provide guidance events to the customer for a correct placement of the carrying arrangement if the customer is identified as having goods in his hands, i.e., abnormal behavior marked as needing guidance includes a plurality of products being in the customer's hand);
"a situation where the customer behavior determinator recognizes one product in the customer's hand, but the recognized product is not identified"; (Moller, Paras. 71 and 91, teaches tracking the time it takes for a customer to pick up an article, scan it, and place it in the packing area wherein the speed of the action is used to determine behavioral information wherein the behavioral information indicates a probability that the customer failed to scan the article by mistake or negligence, i.e., situation wherein customer behavior recognizes a product in the customer's hand but the recognized product was not scanned or identified);
"and a situation where the product picked up by the customer from the pre-identification product table is not identified and moved to the post-identification product table"; (Moller, Para. 108, teaches the behavioral information determining the likelihood of the customer being likely to steal by mistaking or forgettingly placing un-scanned goods in the packing area, i.e., a situation where the product picked up by the customer from the pre-identification product table is not identified and moved to the post-identification product table).
Regarding Claim 7, the combination of references of Moller in view of Johnson and Gabale teaches "The self-checkout apparatus of claim 6, wherein the pre-identification product table includes a first weighing sensor that measures the weight of products placed on the pre-identification product table and the post-identification product table includes a second weighing sensor that measures the weight of products placed on the post-identification product table"; (Moller, Fig. 5 and Paras. 40, 127, and 129, teaches utilizing weight sensors which collect the weight of the shopping basket by means of a weight sensor in the picking area and a weight reported by the weight sensor of the packing area, i.e., pre-identification table includes a first weighing sensor that measures the weight of products and the post-identification table includes a second weighing sensor that measures the weight of products placed on the post-identification table);
"and wherein, as the first or the second weighing sensor detects a weight change in at least one of the pre-identification product table and the post-identification product table and the product is not identified by the product identifier at the time of detecting the weight change or if the weighing sensor detects a smaller weight increase in the post-identification product table than the weight decrease in the pre-identification product table and the product is identified by the product identifier at the time of detecting the weight change, the processor outputs a specified result to decide that the customer has committed an abnormal behavior"; (Moller, Paras. 91 and 129-139, teaches the behavioral analysis module estimating risk of shoplifted items based on the difference between visually estimated weight and weight reported by the weight sensor of the packing area and wherein the behavioral analysis module determines that all articles are correctly scanned when the change in weight from the picking area to the scan area to the packing area is essentially the same as the change in weight on the packing area increase is the same as the decrease in weight on the picking area when the item was removed, i.e., weighing sensors detect a weight change in the pre-identification table and the post-identification table wherein the behavioral analysis module would output guidance if an article was not correctly scanned and estimate risk of shoplifted items indicating abnormal behavior).
Claim 13 recites a method with steps corresponding to the elements of the apparatus recited in Claim 2. Therefore, the recited steps of this claim are mapped to the proposed combination in the same manner as the corresponding elements in its corresponding apparatus claim. Additionally, the rationale and motivation to combine the Moller, Johnson, and Gabale references, presented in rejection of Claim 1, apply to this claim.
Claim 14 recites a method with steps corresponding to the elements of the apparatus recited in Claim 3. Therefore, the recited steps of this claim are mapped to the proposed combination in the same manner as the corresponding elements in its corresponding apparatus claim. Additionally, the rationale and motivation to combine the Moller, Johnson, and Gabale references, presented in rejection of Claim 1, apply to this claim.
Claim 17 recites a method with steps corresponding to the elements of the apparatus recited in Claim 6. Therefore, the recited steps of this claim are mapped to the proposed combination in the same manner as the corresponding elements in its corresponding apparatus claim. Additionally, the rationale and motivation to combine the Moller, Johnson, and Gabale references, presented in rejection of Claim 1, apply to this claim.
Claim 18 recites a method with steps corresponding to the elements of the apparatus recited in Claim 7. Therefore, the recited steps of this claim are mapped to the proposed combination in the same manner as the corresponding elements in its corresponding apparatus claim. Additionally, the rationale and motivation to combine the Moller, Johnson, and Gabale references, presented in rejection of Claim 1, apply to this claim.
Claim 22 recites an apparatus with elements corresponding to the elements and steps recited in Claim 2. Therefore, the recited elements of this claim are mapped to the proposed combination in the same manner as the corresponding steps in its corresponding method claim. Additionally, the rationale and motivation to combine the Moller, Johnson, and Gabale references, presented in rejection of Claim 2, apply to this claim. Finally, the combination of the Moller, Johnson, and Gabale references discloses a processor and a camera with a single photographing direction aligned to the product identification zone(for example, see Johnson, Paragraphs 12 and 17).
Claims 4 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Moller in view of Johnson, Gabale, and Handshaw (US 20210097517 A1).
Regarding Claim 4, the combination of references of Moller in view of Johnson and Gabale teaches "The self-checkout apparatus of claim 2, wherein, as a specific product is input, the processor stores a video of the customer's behavior related to the specific product from the video of the customer's behavior captured by the second camera"; (Moller, Para. 43, teaches the sensor controller which collects data from the one or more sensors using memory, i.e., store the video or images of the customer's behavior related to an item from the images or video from the video or images of the customer behavior).
However, the combination of references of Moller in view of Johnson and Gabale does not explicitly teach "and trains the artificial neural network with a video of the customer's behavior related to the specific product, which is specified as an abnormal behavior, from the stored video of the customer's behavior related to the specific product, to update the artificial neural network's ability to determine the customer's abnormal behavior for the specific product".
In an analogous field of endeavor, Handshaw teaches "and trains the artificial neural network with a video of the customer's behavior related to the specific product, which is specified as an abnormal behavior, from the stored video of the customer's behavior related to the specific product, to update the artificial neural network's ability to determine the customer's abnormal behavior for the specific product"; (Handshaw, Para. 50, teaches decode images that do not correspond to the decoded indicia may be stored in a theft-monitoring image set to train the neural network to develop theft classifiers that identify images of an object with certain characteristics and classify the images as attempted theft of the object, i.e., train and update the neural network's ability to determine abnormal behavior for an object as attempted theft with the video or images of the customer's behavior related to the product which is specified as attempted theft or abnormal from a stored video or images related to an object).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Moller, Johnson, and Gabale by including the training of the neural network based on abnormal behavior of a customer related to a product taught by Handshaw. One of ordinary skill in the art would be motivated to combine the references since it increases accuracy of the identification (Handshaw, Para. 5, teaches the motivation of combination to be to increasing accuracy of identifying objects of the neural network).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date.
Claim 15 recites a method with steps corresponding to the elements of the apparatus recited in Claim 4. Therefore, the recited steps of this claim are mapped to the proposed combination in the same manner as the corresponding elements in its corresponding apparatus claim. Additionally, the rationale and motivation to combine the Moller, Johnson, Gabale, and Handshaw references, presented in rejection of Claim 4, apply to this claim.
Claims 5 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Moller in view of Johnson, Gabale, and Murakami et al. (US 20230169833 A1).
Regarding Claim 5, the combination of references of Moller in view of Johnson and Gabale does not explicitly teach "The self-checkout apparatus of claim 2, wherein, as it is determined that the customer has picked up an age-restricted product based on the analysis of the video captured by the second camera, the processor analyzes a face image of the customer who has picked up the age-restricted product to calculate an age of the customer; and as the age of the customer who has picked up the age-restricted product is determined to be below an authorized age for the age-restricted product and the customer who has picked up the age-restricted product is recognized in front of the product recognition table, the processor specifies whether the behavior of the customer who has picked up the age-restricted product is an abnormal behavior based on the identification result of the product captured by the first camera for the age-restricted product and the determination result of the customer's behavior captured by the second camera”.
In an analogous field of endeavor, Murakami teaches "The self-checkout apparatus of claim 2, wherein, as it is determined that the customer has picked up an age-restricted product based on the analysis of the video captured by the second camera, the processor analyzes a face image of the customer who has picked up the age-restricted product to calculate an age of the customer"; (Murakami, Para. 84, teaches acquiring a user's age information specified by face recognition and determine whether the user is subject to age restrictions on purchase of age-restricted items wherein the age verification operation is performed when the items are recognized to include age-restricted items, i.e., customer determined to pick up an age-restricted product wherein the face image of the customer is analyzed to calculate age of the customer);
"and as the age of the customer who has picked up the age-restricted product is determined to be below an authorized age for the age-restricted product and the customer who has picked up the age-restricted product is recognized in front of the product recognition table, the processor specifies whether the behavior of the customer who has picked up the age-restricted product is an abnormal behavior based on the identification result of the product captured by the first camera for the age-restricted product and the determination result of the customer's behavior captured by the second camera"; (Murakami, Fig. 3 and Paras. 105-106, teaches the screen transitioning to an error screen when the user is determined to be subject to age restrictions on purchase of age restricted items, i.e., customer age who picked up the age-restricted product is below the authorized age wherein the customer who picked up the age-restricted product is recognized in front of the product recognition table in which the customer behavior is marked as an error or abnormal based on the identification of the product that is age-restricted and the determination of the customer's age and attempt to purchase an ineligible item).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Moller, Johnson, and Gabale by including the calculation of the age of a customer who picked up an age-restricted item and determining potential abnormal behavior taught by Murakami. One of ordinary skill in the art would be motivated to combine the references since it improves customer convenience (Murakami, Para. 5, teaches the motivation of combination to be to improve customer convenience).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date.
Claim 16 recites a method with steps corresponding to the elements of the apparatus recited in Claim 5. Therefore, the recited steps of this claim are mapped to the proposed combination in the same manner as the corresponding elements in its corresponding apparatus claim. Additionally, the rationale and motivation to combine the Moller, Johnson, Gabale, and Murakami references, presented in rejection of Claim 5, apply to this claim.
Claims 8-9 and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Moller in view of Johnson, Gabale, Carranza et al. (US 20190043207 A1), and Shin et al. (US 20220076023 A1).
Regarding Claim 8, the combination of references of Moller in view of Johnson and Gabale does not explicitly teach "The self-checkout apparatus of claim 2, wherein the processor generates metadata for an event related to the customer's behavior based on the identification result of the product captured by the first camera and the determination result of the customer's behavior captured by the second camera, matches the metadata with video data of the first camera and the second camera from which the metadata is generated to store the resulting data in a video data storage device; and as a keyword is input, searches for the keyword from the metadata to extract and reproduce video data corresponding to the keyword from the video data storage device”.
In an analogous field of endeavor, Carranza teaches "The self-checkout apparatus of claim 1, wherein the processor generates metadata for an event related to the customer's behavior based on the identification result of the product captured by the first camera and the determination result of the customer's behavior captured by the second camera, matches the metadata with video data of the first camera and the second camera from which the metadata is generated to store the resulting data in a video data storage device"; (Carranza, Abstract and Paras. 22, 49, and 54, teaches a processor obtaining metadata associated with a current state of an object such as its behavior wherein the object is captured by a first and second camera and wherein the cameras may store video streams in memory, process the video streams, transmit video streams and associated metadata to another component for processing and storage purposes, i.e., generate metadata related to the event of an object's behavior and match the metadata with the video data of the cameras for storage).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Moller, Johnson, and Gabale wherein a first camera identifies the product and a second camera helps determine behavior by including the generation of metadata related to object behavior and matching the data with the video for storage taught by Carranza. One of ordinary skill in the art would be motivated to combine the references since it improves identification (Carranza, Para. 21, teaches the motivation of combination to be to improve the successful identification of objects).
However, the combination of references of Moller in view of Johnson, Gabale, and Carranza does not explicitly teach "and as a keyword is input, searches for the keyword from the metadata to extract and reproduce video data corresponding to the keyword from the video data storage device".
In an analogous field of endeavor, Shin teaches "and as a keyword is input, searches for the keyword from the metadata to extract and reproduce video data corresponding to the keyword from the video data storage device"; (Shin, Paras. 69 and 74, teaches a user entering one or more search criteria such as keywords to search extracted metadata associated with the video segments, i.e., input a keyword to search for the keyword from the metadata to extract the video data corresponding to the keyword from the video storage).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Moller, Johnson, Gabale, and Carranza by including the searching of an input keyword in the metadata to extract video data corresponding to the keyword taught by Shin. One of ordinary skill in the art would be motivated to combine the references since it eases the process of selecting and browsing video segments (Shin, Para. 69, teaches the motivation of combination to be to make selecting and browsing video segments easier).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date.
Regarding Claim 9, the combination of references of Moller in view of Johnson and Gabale, Carranza, and Shin teaches "The self-checkout apparatus of claim 2, wherein, as the product is recognized from the videos captured by the first camera and the second camera, the processor: stores a video of the customer's behavior related to the recognized product in the video data storage device"; (Gabale, Para. 12, teaches a memory that stores a database and a set of modules wherein the modules comprise video streams of a first and second camera to generate labels and location of at least one object and classify an activity associated with the movement of the product object, i.e., store video of the customer's behavior related to the recognized product);
"records information specifying the recognized product in metadata and then matches the metadata with the video of the customer's behavior related to the recognized product to store the resulting data"; (Carranza, Paras. 49 and 54, teaches cameras that store video streams in memory, process the video streams, transmit video streams and associated metadata to another component for processing and storage purposes wherein the metadata indicates features associated with the object such as object type, characteristics, and behavior, i.e., record information specifying the identified product in the metadata and match it with the video of the object and its behavior for storage);
"and as a keyword specifying the recognized product is input, searches for the keyword from the metadata to extract the video of the customer's behavior related to the recognized product from the video data storage device"; (Shin, Paras. 69 and 74, teaches a user entering one or more search criteria such as keywords to search extracted metadata associated with the video segments wherein the keywords include extractions from audio, visually extracted content, action tags from events, and other extracted features, i.e., input a keyword related to an extracted product to search for the keyword from the metadata to extract the video data corresponding to the keyword from the video storage).
The proposed combination as well as the motivation for combining the Moller, Johnson, Gabale, Carranza, and Shin references presented in the rejection of Claims 1 and 8, applies to claim 9. Thus, the apparatus recited in claim 9 is met by Moller in view of Johnson and Gabale, Carranza, and Shin.
Claim 19 recites a method with steps corresponding to the elements of the apparatus recited in Claim 8. Therefore, the recited steps of this claim are mapped to the proposed combination in the same manner as the corresponding elements in its corresponding apparatus claim. Additionally, the rationale and motivation to combine the Moller, Johnson, Gabale, Carranza, and Shin references, presented in rejection of Claim 8, apply to this claim.
Claim 20 recites a method with steps corresponding to the elements of the apparatus recited in Claim 9. Therefore, the recited steps of this claim are mapped to the proposed combination in the same manner as the corresponding elements in its corresponding apparatus claim. Additionally, the rationale and motivation to combine the Moller, Johnson, Gabale, Carranza, and Shin references, presented in rejection of Claim 8, apply to this claim.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Moller in view of Johnson, Gabale, Carranza, Shin, and Aggarwal et al. (US 20210304291 A1).
Regarding Claim 10, the combination of references of Moller in view of Johnson, Gabale, Carranza, and Shin does not explicitly teach "The self-checkout apparatus of claim 8, wherein the processor generates a personal identification object corresponding to personal identification information of the customer captured by the second camera, and matches the personal identification object with the video data obtained by the first camera related to the customer captured by the second camera and the video data obtained by the second camera to store the resulting data in the video data storage device".
In an analogous field of endeavor, Aggarwal teaches "The self-checkout apparatus of claim 8, wherein the processor generates a personal identification object corresponding to personal identification information of the customer captured by the second camera, and matches the personal identification object with the video data obtained by the first camera related to the customer captured by the second camera and the video data obtained by the second camera to store the resulting data in the video data storage device"; (Aggarwal, Paras. 25 and 75-78, teaches identifying customers by one or more cameras by corresponding a unique membership identification corresponding to the face wherein the customer identifier is obtained and a corresponding customer profile wherein the cameras are coupled with a memory storage device, i.e., generate a personal identification object corresponding to personal identification information of the customer captured by cameras and match the personal identification object with the video of the customer obtained by the cameras for storage).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Moller, Johnson, Gabale, Carranza, and Shin by including the generation of a personal identification object corresponding to the personal identification information of the customer and matching the personal identification object with the video data for storage taught by Aggarwal. One of ordinary skill in the art would be motivated to combine the references since it improves the operation of the system and enhances reliability (Aggarwal, Para. 74, teaches the motivation of combination to be to improve the operation of the system, provide redundancy, and enhance reliability).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Moller in view of Johnson, Gabale, and Suzuki (US 20230034988 A1).
Regarding Claim 11, the combination of references of Moller in view of Johnson and Gabale does not explicitly teach "The self-checkout apparatus of claim 2, wherein the artificial neural network determines the customer's behavior by utilizing a skeleton algorithm for the customer in the video of the customer's behavior captured by the second camera".
In an analogous field of endeavor, Suzuki teaches "The self-checkout apparatus of claim 2, wherein the artificial neural network determines the customer's behavior by utilizing a skeleton algorithm for the customer in the video of the customer's behavior captured by the second camera"; (Suzuki, Fig. 8 and Paras. 76 and 80, teaches processing and evaluating a person's behavior wherein a neural network is used which performs skeleton detection of the person, i.e., artificial neural network determines behavior by utilizing a skeleton algorithm for the customer).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Moller, Johnson, and Gabale by including the use of a skeleton algorithm with the neural network taught by Suzuki. One of ordinary skill in the art would be motivated to combine the references since it improves the accuracy of detected fraudulent behavior (Suzuki, Para. 56, teaches the motivation of combination to be to improve accuracy of detecting a fraudulent behavior at the time of product purchase by self-scan).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDREW STEVEN BUDISALICH whose telephone number is (703)756-5568. The examiner can normally be reached Monday - Friday 8:30am-5:00pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached on (571) 272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANDREW S BUDISALICH/Examiner, Art Unit 2662
/AMANDEEP SAINI/Supervisory Patent Examiner, Art Unit 2662
Prosecution Insights