DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 06/25/2026 has been entered.
Status of Claims
Claims 2-11, 13-20, and 22 are pending. Claims 1, 12, and 21 are canceled.
Response to Arguments
Applicant’s arguments, see p.22-25, filed 09/10/2026, with respect to the interpretation of Claims 2-11 and 22 under 35 U.S.C. 112(f) have been fully considered but are not persuasive. Applicant argues that the limitations of the claims recite sufficient structure or acts to perform the claimed function and supplying evidence of the structure found in the specification. Examiner disagrees because interpreting Claims 2-11 and 22 under 35 U.S.C. 112(f) means that the given limitations within the claims are not given sufficient structure within the claim language and thus requires the limitation to be interpreted to cover the corresponding structure described in the specification as performing the claimed function.
Applicant’s arguments, see p.25-31, filed 09/10/2026, with respect to the rejections of Claims 2-11, 13-20, and 22 under 35 U.S.C. 103 have been fully considered and are persuasive. However, new grounds of rejection are presented below.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
Use of the word “means” (or “step for”) in a claim with functional language creates a rebuttable presumption that the claim element is to be treated in accordance with 35 U.S.C. 112(f) (pre-AIA 35 U.S.C. 112, sixth paragraph). The presumption that 35 U.S.C. 112(f) (pre-AIA 35 U.S.C. 112, sixth paragraph) is invoked is rebutted when the function is recited with sufficient structure, material, or acts within the claim itself to entirely perform the recited function.
Absence of the word “means” (or “step for”) in a claim creates a rebuttable presumption that the claim element is not to be treated in accordance with 35 U.S.C. 112(f) (pre-AIA 35 U.S.C. 112, sixth paragraph). The presumption that 35 U.S.C. 112(f) (pre-AIA 35 U.S.C. 112, sixth paragraph) is not invoked is rebutted when the claim element recites function but fails to recite sufficiently definite structure, material or acts to perform that function.
Claim elements in this application that use the word “means” (or “step for”) are presumed to invoke 35 U.S.C. 112(f) except as otherwise indicated in an Office action. Similarly, claim elements that do not use the word “means” (or “step for”) are presumed not to invoke 35 U.S.C. 112(f) except as otherwise indicated in an Office action.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are: “a product identifier” and “a customer behavior determinator” relevant to Claims 2-11 and 22.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 2-3, 6-7, 13-14, 17-18 are rejected under 35 U.S.C. 103 as being unpatentable over Moller et al. (US 20230058903 A1) in view of Johnson et al. (US 20230186266 A1), Gabale (US 20240169735 A1), and Grigg et al. (US 20150294385 A1).
Regarding Claim 2, Moller teaches "A self-checkout apparatus comprising: a pre-identification product table (21); a product recognition table (11); a post-identification product table (31); the product recognition table (11) being provided with a product identification zone (11-1) and on which a product to be identified is located"; (Moller, Fig. 5 and Paras. 31 and 97-100, teaches a self-checkout system comprising a first area, a second area, and a third area wherein a first area may be seen as an unpacking area, a second area which may be seen as a scanning area arranged with an identification means configured to identify the articles wherein the scanning means is in the form of a barcode reader, and a third area may be seen as a packing area where the customer is placing its identified articles , i.e., a self-checkout apparatus comprising a pre-identification product table being the first area packing area as well as a product recognition table with an identification zone for identifying products being the second area with a scanning means and a post-identification product table being the third area packing area);
"
"
"
"
"a customer behavior determinator which interprets the video of the surveillance area obtained by the second camera to output a determination result of a customer's behavior captured by the second camera"; (Moller, Paras. 5 and 40, teaches a customer behavioral system comprising a sensor arrangement with one or more sensors and a behavioral analysis module configured to determine at least one behavioral feature of a customer based on sensor data from said one or more sensors to determine behavioral information, i.e., customer behavior determinator which interprets the video of the surveillance area of a second camera sensor to output a determination result of the customer's behavior captured by the second camera);
"a processor which specifies the customer's behavior as an abnormal behavior and a normal behavior based on the identification result of the product captured by the first camera and the determination result of the customer's behavior captured by the second camera, and generates a control signal depending on the specified result"; (Moller, Paras. 85-87, teaches the behavior system using behavioral information to detect if a customer is using the checkout counter the right way wherein the system will not prompt or guide the customer further or if the customer is acting in a deviant way wherein an output means may transmit a signal to the checkout system in the store or transmit instructions to execute an anti-theft operation, i.e., a processor that specifies the customer's behavior as normal being the correct way of operation and abnormal as deviant in which a control signal is generated based on the result);
"and wherein the customer behavior determinator comprises an artificial neural network which analyzes the video of the surveillance area obtained by the second camera to determine the customer's behavior captured by the second camera"; (Moller, Paras. 22 and 45, teaches using convolutional neural networks to process the data gathered from the at least one sensor or plurality of sensors to determine behavioral information, i.e., customer behavior determinator comprises an artificial neural network to analyze the video of the area to determine customer's behavior);
"and wherein the artificial neural network is trained by learning a video of the customer's behavior captured by the second camera as the product is identified by the product identifier and a video of the customer's behavior captured by the second camera as the product is not identified by the product identifier to determine the customer's behavior that does not correspond to the behavior of locating a product in the product identification zone to be identified"; (Moller, Paras. 77-78, 91 and 109, teaches behavioral information being used to determine and output statistical data which is used to learn and update the system by using the statistical data as an input when determining future behavioral information wherein behavioral information includes determining the probability level related to the likelihood a customer will steal based on a failure to scan one or more articles, i.e., neural network is trained using video of customer behavior captured by the camera sensor as the product is identified by the product identifier and using video of customer behavior when the product is not identified by the product identifier to determine customer behavior that does not correspond to the behavior of locating a product in the product identification zone to be identified that is the likelihood of the customer stealing or leaving un-scanned items as the abnormal behavior).
However, Moller does not explicitly teach "a first camera having its photographing area defined by the product identification zone (11-1) of the product recognition table (11); the first camera obtaining a video of the product in the product identification zone (11-1) by photographing the product identification zone (11-1); a second camera which obtains a video of a surveillance area by photographing the surveillance area including the product recognition table (11) and the inside of a store; a product identifier which detects an identification code assigned to the product from the video of the product obtained by the first camera and interprets the detected identification code to output an identification result of the product captured by the first camera".
In an analogous field of endeavor, Johnson teaches "a first camera having its photographing area defined by the product identification zone (11-1) of the product recognition table (11); (Johnson, FIG. 2A and Paras. 17-18, teaches a self-checkout kiosk which includes a scan area which may include a scanner scale which can weigh and scan items wherein the kiosk also includes a camera that has a field of view defined on the scan area wherein Figure 2A uses dashed lines to define the field of view of the camera as around the scanner scale area used to scan items, i.e., first camera has its photographing area defined by the product identification zone of the recognition table being the camera field of view defined by the scanner scale area of the scan area);
"the first camera obtaining a video of the product in the product identification zone (11-1) by photographing the product identification zone (11-1)"; (Johnson, FIG. 2A and Para. 37, teaches the camera may continuously capture video in the scanning area to detect at least one target object , i.e., first camera obtains video of the product in the identification zone by photographing the identification zone).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Moller by including the camera having a photographing area defined by the identification zone for obtaining video of the product taught by Johnson. One of ordinary skill in the art would be motivated to combine the references since it improves measurement processing (Johnson, Para. 11, teaches the motivation of combination to be to improve techniques for performing measurement information processing).
However, the combination of references of Moller in view of Johnson does not explicitly teach "a second camera which obtains a video of a surveillance area by photographing the surveillance area including the product recognition table (11) and the inside of a store; a product identifier which detects an identification code assigned to the product from the video of the product obtained by the first camera and interprets the detected identification code to output an identification result of the product captured by the first camera ".
In an analogous field of endeavor, Gabale teaches "a second camera which obtains a video of a surveillance area by photographing the surveillance area including the product recognition table (11) and the inside of a store"; (Gabale, Fig. 3 and Para. 6, teaches configuring a camera at the self-checkout counter or on a surface of the physical retail store environment wherein a second camera video stream reidentifies the product object in association with the at least one person object by tracking movement of the product object in the physical retail store environment, i.e., a second camera obtains video of a surveillance area by photographing the surveillance area including the product recognition table and the inside of a store).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Moller and Johnson by including a second camera to obtain video of a surveillance area within the store including the product recognition table taught by Gabale. One of ordinary skill in the art would be motivated to combine the references since it prevents shrinkages in a retail environment (Gabale, Para. 1, teaches the motivation of combination to be to use real-time camera feeds to prevent shrinkages in a retail environment).
However, the combination of references of Moller in view of Johson and Gabale does not explicitly teach "a product identifier which detects an identification code assigned to the product from the video of the product obtained by the first camera and interprets the detected identification code to output an identification result of the product captured by the first camera".
In an analogous field of endeavor, Grigg teaches "a product identifier which detects an identification code assigned to the product from the video of the product obtained by the first camera and interprets the detected identification code to output an identification result of the product captured by the first camera"; (Grigg, Paras. 66, 68, and 80, teaches the item identification application is configured to determine that one or more images in the video stream identify the items wherein the identity of the items may be determined by capturing and reading indicia of an item or capturing and reading an image-based code such as a barcode or QR code located on or proximate to the item, i.e., identifier detects an identification code assigned to the product from the captured video obtained from a camera and interpret that code to output the identification result of the captured product).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Moller, Johnson, and Gabale by including the detecting of an identification code assigned to the product from the video of the product and output the result taught by Grigg. One of ordinary skill in the art would be motivated to combine the references since it automatically and efficiently provides information to the user (Grigg, Abstract, teaches the motivation of combination to be to automatically provide information to the user in a highly efficient manner).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date.
Regarding Claim 3, the combination of references of Moller in view of Johnson, Gabale, and Grigg teaches "The self-checkout apparatus of claim 2, wherein the processor analyzes the video of the surveillance area obtained by the second camera to specify the customer's behavior captured by the second camera as an abnormal behavior and a normal behavior based on at least one of a change in facial expression, a change in face color, and a heart rate reading of the customer captured by the second camera, and generates a control signal depending on the specified result"; (Moller, Paras. 53-58 and 82, teaches behavioral features used to determine behavioral information including facial expressions, analysis of facial coloration of the customer, and an approximation of the heart rate of the customer wherein a communication signal may be transmitted based on the determined behavioral information, i.e., determine behavioral information as normal or abnormal based on changes in facial expression, face color, and heart rate of the customer and generate a control signal depending on the result).
Regarding Claim 6, the combination of references of Moller in view of Johnson, Gabale, and Grigg teaches "The self-checkout apparatus of claim 2, further comprising a pre-identification product table and a post-identification product table disposed in a predetermined area around the product recognition table, wherein the abnormal behavior of the customer include: a situation where the customer behavior determinator recognizes two or more products in the customer's hand"; (Moller, Fig. 5 and Paras. 98-100 and 118-120, teaches a first area which is an unpacking area where the customer places his articles that are to be purchased and a third area which is a packing area for the customer to place its identified articles, i.e., pre-identification product table and a post-identification product table disposed in a predetermined area around the product recognition table, and wherein the behavioral analysis module may provide guidance events to the customer for a correct placement of the carrying arrangement if the customer is identified as having goods in his hands, i.e., abnormal behavior marked as needing guidance includes a plurality of products being in the customer's hand);
"a situation where the customer behavior determinator recognizes one product in the customer's hand, but the recognized product is not identified"; (Moller, Paras. 71 and 91, teaches tracking the time it takes for a customer to pick up an article, scan it, and place it in the packing area wherein the speed of the action is used to determine behavioral information wherein the behavioral information indicates a probability that the customer failed to scan the article by mistake or negligence, i.e., situation wherein customer behavior recognizes a product in the customer's hand but the recognized product was not scanned or identified);
"and a situation where the product picked up by the customer from the pre-identification product table is not identified and moved to the post-identification product table"; (Moller, Para. 108, teaches the behavioral information determining the likelihood of the customer being likely to steal by mistaking or forgettingly placing un-scanned goods in the packing area, i.e., a situation where the product picked up by the customer from the pre-identification product table is not identified and moved to the post-identification product table).
Regarding Claim 7, the combination of references of Moller in view of Johnson, Gabale, and Grigg teaches "The self-checkout apparatus of claim 6, wherein the pre-identification product table includes a first weighing sensor that measures the weight of products placed on the pre-identification product table and the post-identification product table includes a second weighing sensor that measures the weight of products placed on the post-identification product table"; (Moller, Fig. 5 and Paras. 40, 127, and 129, teaches utilizing weight sensors which collect the weight of the shopping basket by means of a weight sensor in the picking area and a weight reported by the weight sensor of the packing area, i.e., pre-identification table includes a first weighing sensor that measures the weight of products and the post-identification table includes a second weighing sensor that measures the weight of products placed on the post-identification table);
"and wherein, as the first or the second weighing sensor detects a weight change in at least one of the pre-identification product table and the post-identification product table and the product is not identified by the product identifier at the time of detecting the weight change or if the weighing sensor detects a smaller weight increase in the post-identification product table than the weight decrease in the pre-identification product table and the product is identified by the product identifier at the time of detecting the weight change, the processor outputs a specified result to decide that the customer has committed an abnormal behavior"; (Moller, Paras. 91 and 129-139, teaches the behavioral analysis module estimating risk of shoplifted items based on the difference between visually estimated weight and weight reported by the weight sensor of the packing area and wherein the behavioral analysis module determines that all articles are correctly scanned when the change in weight from the picking area to the scan area to the packing area is essentially the same as the change in weight on the packing area increase is the same as the decrease in weight on the picking area when the item was removed, i.e., weighing sensors detect a weight change in the pre-identification table and the post-identification table wherein the behavioral analysis module would output guidance if an article was not correctly scanned and estimate risk of shoplifted items indicating abnormal behavior).
Claim 13 recites a method with steps corresponding to the elements of the apparatus recited in Claim 2. Therefore, the recited steps of this claim are mapped to the proposed combination in the same manner as the corresponding elements in its corresponding apparatus claim. Additionally, the rationale and motivation to combine the Moller, Johnson, Gabale, and Grigg references, presented in rejection of Claim 1, apply to this claim.
Claim 14 recites a method with steps corresponding to the elements of the apparatus recited in Claim 3. Therefore, the recited steps of this claim are mapped to the proposed combination in the same manner as the corresponding elements in its corresponding apparatus claim. Additionally, the rationale and motivation to combine the Moller, Johnson, Gabale, and Grigg references, presented in rejection of Claim 1, apply to this claim.
Claim 17 recites a method with steps corresponding to the elements of the apparatus recited in Claim 6. Therefore, the recited steps of this claim are mapped to the proposed combination in the same manner as the corresponding elements in its corresponding apparatus claim. Additionally, the rationale and motivation to combine the Moller, Johnson, Gabale, and Grigg references, presented in rejection of Claim 1, apply to this claim.
Claim 18 recites a method with steps corresponding to the elements of the apparatus recited in Claim 7. Therefore, the recited steps of this claim are mapped to the proposed combination in the same manner as the corresponding elements in its corresponding apparatus claim. Additionally, the rationale and motivation to combine the Moller, Johnson, Gabale, and Grigg references, presented in rejection of Claim 1, apply to this claim.
Claims 4 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Moller in view of Johnson, Gabale, Grigg, and Handshaw (US 20210097517 A1).
Regarding Claim 4, the combination of references of Moller in view of Johnson, Gabale, and Grigg teaches "The self-checkout apparatus of claim 2, wherein, as a specific product is input, the processor stores a video of the customer's behavior related to the specific product from the video of the customer's behavior captured by the second camera"; (Moller, Para. 43, teaches the sensor controller which collects data from the one or more sensors using memory, i.e., store the video or images of the customer's behavior related to an item from the images or video from the video or images of the customer behavior).
However, the combination of references of Moller in view of Johnson, Gabale, and Grigg does not explicitly teach "and trains the artificial neural network with a video of the customer's behavior related to the specific product, which is specified as an abnormal behavior, from the stored video of the customer's behavior related to the specific product, to update the artificial neural network's ability to determine the customer's abnormal behavior for the specific product".
In an analogous field of endeavor, Handshaw teaches "and trains the artificial neural network with a video of the customer's behavior related to the specific product, which is specified as an abnormal behavior, from the stored video of the customer's behavior related to the specific product, to update the artificial neural network's ability to determine the customer's abnormal behavior for the specific product"; (Handshaw, Para. 50, teaches decode images that do not correspond to the decoded indicia may be stored in a theft-monitoring image set to train the neural network to develop theft classifiers that identify images of an object with certain characteristics and classify the images as attempted theft of the object, i.e., train and update the neural network's ability to determine abnormal behavior for an object as attempted theft with the video or images of the customer's behavior related to the product which is specified as attempted theft or abnormal from a stored video or images related to an object).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Moller, Johnson, Gabale, and Grigg by including the training of the neural network based on abnormal behavior of a customer related to a product taught by Handshaw. One of ordinary skill in the art would be motivated to combine the references since it increases accuracy of the identification (Handshaw, Para. 5, teaches the motivation of combination to be to increasing accuracy of identifying objects of the neural network).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date.
Claim 15 recites a method with steps corresponding to the elements of the apparatus recited in Claim 4. Therefore, the recited steps of this claim are mapped to the proposed combination in the same manner as the corresponding elements in its corresponding apparatus claim. Additionally, the rationale and motivation to combine the Moller, Johnson, Gabale, Grigg, and Handshaw references, presented in rejection of Claim 4, apply to this claim.
Claims 5 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Moller in view of Johnson, Gabale, Grigg, and Murakami et al. (US 20230169833 A1).
Regarding Claim 5, the combination of references of Moller in view of Johnson, Gabale, and Grigg does not explicitly teach "The self-checkout apparatus of claim 2, wherein, as it is determined that the customer has picked up an age-restricted product based on the analysis of the video captured by the second camera, the processor analyzes a face image of the customer who has picked up the age-restricted product to calculate an age of the customer; and as the age of the customer who has picked up the age-restricted product is determined to be below an authorized age for the age-restricted product and the customer who has picked up the age-restricted product is recognized in front of the product recognition table, the processor specifies whether the behavior of the customer who has picked up the age-restricted product is an abnormal behavior based on the identification result of the product captured by the first camera for the age-restricted product and the determination result of the customer's behavior captured by the second camera”.
In an analogous field of endeavor, Murakami teaches "The self-checkout apparatus of claim 2, wherein, as it is determined that the customer has picked up an age-restricted product based on the analysis of the video captured by the second camera, the processor analyzes a face image of the customer who has picked up the age-restricted product to calculate an age of the customer"; (Murakami, Para. 84, teaches acquiring a user's age information specified by face recognition and determine whether the user is subject to age restrictions on purchase of age-restricted items wherein the age verification operation is performed when the items are recognized to include age-restricted items, i.e., customer determined to pick up an age-restricted product wherein the face image of the customer is analyzed to calculate age of the customer);
"and as the age of the customer who has picked up the age-restricted product is determined to be below an authorized age for the age-restricted product and the customer who has picked up the age-restricted product is recognized in front of the product recognition table, the processor specifies whether the behavior of the customer who has picked up the age-restricted product is an abnormal behavior based on the identification result of the product captured by the first camera for the age-restricted product and the determination result of the customer's behavior captured by the second camera"; (Murakami, Fig. 3 and Paras. 105-106, teaches the screen transitioning to an error screen when the user is determined to be subject to age restrictions on purchase of age restricted items, i.e., customer age who picked up the age-restricted product is below the authorized age wherein the customer who picked up the age-restricted product is recognized in front of the product recognition table in which the customer behavior is marked as an error or abnormal based on the identification of the product that is age-restricted and the determination of the customer's age and attempt to purchase an ineligible item).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Moller, Johnson, Gabale, and Grigg by including the calculation of the age of a customer who picked up an age-restricted item and determining potential abnormal behavior taught by Murakami. One of ordinary skill in the art would be motivated to combine the references since it improves customer convenience (Murakami, Para. 5, teaches the motivation of combination to be to improve customer convenience).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date.
Claim 16 recites a method with steps corresponding to the elements of the apparatus recited in Claim 5. Therefore, the recited steps of this claim are mapped to the proposed combination in the same manner as the corresponding elements in its corresponding apparatus claim. Additionally, the rationale and motivation to combine the Moller, Johnson, Gabale, Grigg, and Murakami references, presented in rejection of Claim 5, apply to this claim.
Claims 8-9 and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Moller in view of Johnson, Gabale, Grigg, Carranza et al. (US 20190043207 A1), and Shin et al. (US 20220076023 A1).
Regarding Claim 8, the combination of references of Moller in view of Johnson, Gabale, and Grigg does not explicitly teach "The self-checkout apparatus of claim 2, wherein the processor generates metadata for an event related to the customer's behavior based on the identification result of the product captured by the first camera and the determination result of the customer's behavior captured by the second camera, matches the metadata with video data of the first camera and the second camera from which the metadata is generated to store the resulting data in a video data storage device; and as a keyword is input, searches for the keyword from the metadata to extract and reproduce video data corresponding to the keyword from the video data storage device”.
In an analogous field of endeavor, Carranza teaches "The self-checkout apparatus of claim 1, wherein the processor generates metadata for an event related to the customer's behavior based on the identification result of the product captured by the first camera and the determination result of the customer's behavior captured by the second camera, matches the metadata with video data of the first camera and the second camera from which the metadata is generated to store the resulting data in a video data storage device"; (Carranza, Abstract and Paras. 22, 49, and 54, teaches a processor obtaining metadata associated with a current state of an object such as its behavior wherein the object is captured by a first and second camera and wherein the cameras may store video streams in memory, process the video streams, transmit video streams and associated metadata to another component for processing and storage purposes, i.e., generate metadata related to the event of an object's behavior and match the metadata with the video data of the cameras for storage).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Moller, Johnson, Gabale, and Grigg wherein a first camera identifies the product and a second camera helps determine behavior by including the generation of metadata related to object behavior and matching the data with the video for storage taught by Carranza. One of ordinary skill in the art would be motivated to combine the references since it improves identification (Carranza, Para. 21, teaches the motivation of combination to be to improve the successful identification of objects).
However, the combination of references of Moller in view of Johnson, Gabale, Grigg, and Carranza does not explicitly teach "and as a keyword is input, searches for the keyword from the metadata to extract and reproduce video data corresponding to the keyword from the video data storage device".
In an analogous field of endeavor, Shin teaches "and as a keyword is input, searches for the keyword from the metadata to extract and reproduce video data corresponding to the keyword from the video data storage device"; (Shin, Paras. 69 and 74, teaches a user entering one or more search criteria such as keywords to search extracted metadata associated with the video segments, i.e., input a keyword to search for the keyword from the metadata to extract the video data corresponding to the keyword from the video storage).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Moller, Johnson, Gabale, Grigg, and Carranza by including the searching of an input keyword in the metadata to extract video data corresponding to the keyword taught by Shin. One of ordinary skill in the art would be motivated to combine the references since it eases the process of selecting and browsing video segments (Shin, Para. 69, teaches the motivation of combination to be to make selecting and browsing video segments easier).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date.
Regarding Claim 9, the combination of references of Moller in view of Johnson, Gabale, Grigg, Carranza, and Shin teaches "The self-checkout apparatus of claim 2, wherein, as the product is recognized from the videos captured by the first camera and the second camera, the processor: stores a video of the customer's behavior related to the recognized product in the video data storage device"; (Gabale, Para. 12, teaches a memory that stores a database and a set of modules wherein the modules comprise video streams of a first and second camera to generate labels and location of at least one object and classify an activity associated with the movement of the product object, i.e., store video of the customer's behavior related to the recognized product);
"records information specifying the recognized product in metadata and then matches the metadata with the video of the customer's behavior related to the recognized product to store the resulting data"; (Carranza, Paras. 49 and 54, teaches cameras that store video streams in memory, process the video streams, transmit video streams and associated metadata to another component for processing and storage purposes wherein the metadata indicates features associated with the object such as object type, characteristics, and behavior, i.e., record information specifying the identified product in the metadata and match it with the video of the object and its behavior for storage);
"and as a keyword specifying the recognized product is input, searches for the keyword from the metadata to extract the video of the customer's behavior related to the recognized product from the video data storage device"; (Shin, Paras. 69 and 74, teaches a user entering one or more search criteria such as keywords to search extracted metadata associated with the video segments wherein the keywords include extractions from audio, visually extracted content, action tags from events, and other extracted features, i.e., input a keyword related to an extracted product to search for the keyword from the metadata to extract the video data corresponding to the keyword from the video storage).
The proposed combination as well as the motivation for combining the Moller, Johnson, Gabale, Grigg, Carranza, and Shin references presented in the rejection of Claims 1 and 8, applies to claim 9. Thus, the apparatus recited in claim 9 is met by Moller in view of Johnson, Gabale, and Grigg, Carranza, and Shin.
Claim 19 recites a method with steps corresponding to the elements of the apparatus recited in Claim 8. Therefore, the recited steps of this claim are mapped to the proposed combination in the same manner as the corresponding elements in its corresponding apparatus claim. Additionally, the rationale and motivation to combine the Moller, Johnson, Gabale, Grigg, Carranza, and Shin references, presented in rejection of Claim 8, apply to this claim.
Claim 20 recites a method with steps corresponding to the elements of the apparatus recited in Claim 9. Therefore, the recited steps of this claim are mapped to the proposed combination in the same manner as the corresponding elements in its corresponding apparatus claim. Additionally, the rationale and motivation to combine the Moller, Johnson, Gabale, Grigg, Carranza, and Shin references, presented in rejection of Claim 8, apply to this claim.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Moller in view of Johnson, Gabale, Grigg, Carranza, Shin, and Aggarwal et al. (US 20210304291 A1).
Regarding Claim 10, the combination of references of Moller in view of Johnson, Gabale, Grigg, Carranza, and Shin does not explicitly teach "The self-checkout apparatus of claim 8, wherein the processor generates a personal identification object corresponding to personal identification information of the customer captured by the second camera, and matches the personal identification object with the video data obtained by the first camera related to the customer captured by the second camera and the video data obtained by the second camera to store the resulting data in the video data storage device".
In an analogous field of endeavor, Aggarwal teaches "The self-checkout apparatus of claim 8, wherein the processor generates a personal identification object corresponding to personal identification information of the customer captured by the second camera, and matches the personal identification object with the video data obtained by the first camera related to the customer captured by the second camera and the video data obtained by the second camera to store the resulting data in the video data storage device"; (Aggarwal, Paras. 25 and 75-78, teaches identifying customers by one or more cameras by corresponding a unique membership identification corresponding to the face wherein the customer identifier is obtained and a corresponding customer profile wherein the cameras are coupled with a memory storage device, i.e., generate a personal identification object corresponding to personal identification information of the customer captured by cameras and match the personal identification object with the video of the customer obtained by the cameras for storage).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Moller, Johnson, Gabale, Grigg, Carranza, and Shin by including the generation of a personal identification object corresponding to the personal identification information of the customer and matching the personal identification object with the video data for storage taught by Aggarwal. One of ordinary skill in the art would be motivated to combine the references since it improves the operation of the system and enhances reliability (Aggarwal, Para. 74, teaches the motivation of combination to be to improve the operation of the system, provide redundancy, and enhance reliability).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Moller in view of Johnson, Gabale, Grigg, and Suzuki (US 20230034988 A1).
Regarding Claim 11, the combination of references of Moller in view of Johnson, Gabale, and Grigg does not explicitly teach "The self-checkout apparatus of claim 2, wherein the artificial neural network determines the customer's behavior by utilizing a skeleton algorithm for the customer in the video of the customer's behavior captured by the second camera".
In an analogous field of endeavor, Suzuki teaches "The self-checkout apparatus of claim 2, wherein the artificial neural network determines the customer's behavior by utilizing a skeleton algorithm for the customer in the video of the customer's behavior captured by the second camera"; (Suzuki, Fig. 8 and Paras. 76 and 80, teaches processing and evaluating a person's behavior wherein a neural network is used which performs skeleton detection of the person, i.e., artificial neural network determines behavior by utilizing a skeleton algorithm for the customer).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Moller, Johnson, Gabale, and Grigg by including the use of a skeleton algorithm with the neural network taught by Suzuki. One of ordinary skill in the art would be motivated to combine the references since it improves the accuracy of detected fraudulent behavior (Suzuki, Para. 56, teaches the motivation of combination to be to improve accuracy of detecting a fraudulent behavior at the time of product purchase by self-scan).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date.
Claim 22 is rejected under 35 U.S.C. 103 as being unpatentable over Moller in view of Johnson, Gabale, Grigg, and Crooks et al. (US 20180288300 A1).
Regarding Claim 22, the combination of references of Moller in view of Johnson, Gabale, and Grigg teaches "A self-checkout apparatus comprising: a pre-identification product table (21); a product recognition table (11); a post-identification product table (31); the product recognition table (11) being provided with a product identification zone (11-1) and on which a product to be identified is located"; (Moller, Fig. 5 and Paras. 31 and 97-100, teaches a self-checkout system comprising a first area, a second area, and a third area wherein a first area may be seen as an unpacking area, a second area which may be seen as a scanning area arranged with an identification means configured to identify the articles wherein the scanning means is in the form of a barcode reader, and a third area may be seen as a packing area where the customer is placing its identified articles , i.e., a self-checkout apparatus comprising a pre-identification product table being the first area packing area as well as a product recognition table with an identification zone for identifying products being the second area with a scanning means and a post-identification product table being the third area packing area);
"";
"the first camera which is arranged so that its single photographing direction is aligned to the product identification zone (11-1) and which obtains a video of the product in the product identification zone (11-1) by photographing the product identification zone (11-1)"; (Johnson, FIG. 2A and Paras. 12, 17, and 37, teaches the self-checkout kiosk includes a camera that has a field of view of the scan area, i.e., camera arranged with single photographing direction aligned to the product identification zone, and wherein the camera may continuously capture video in the scanning area to detect at least one target object , i.e., first camera obtains video of the product in the identification zone by photographing the identification zone);
"the second camera which obtains a video of a surveillance area by photographing the surveillance area including the product recognition table (11) and the inside of a store"; (Gabale, Fig. 3 and Para. 6, teaches configuring a camera at the self-checkout counter or on a surface of the physical retail store environment wherein a second camera video stream reidentifies the product object in association with the at least one person object by tracking movement of the product object in the physical retail store environment, i.e., a second camera obtains video of a surveillance area by photographing the surveillance area including the product recognition table and the inside of a store);
"a product identifier which detects an identification code assigned to the product from the video of the product obtained by the first camera and interprets the detected identification code to output an identification result of the product captured by the first camera"; (Grigg, Paras. 66, 68, and 80, teaches the item identification application is configured to determine that one or more images in the video stream identify the items wherein the identity of the items may be determined by capturing and reading indicia of an item or capturing and reading an image-based code such as a barcode or QR code located on or proximate to the item, i.e., identifier detects an identification code assigned to the product from the captured video obtained from a camera and interpret that code to output the identification result of the captured product);
"a customer behavior determinator which interprets the video of the surveillance area obtained by the second camera to output a determination result of a customer's behavior captured by the second camera"; (Moller, Paras. 5 and 40, teaches a customer behavioral system comprising a sensor arrangement with one or more sensors and a behavioral analysis module configured to determine at least one behavioral feature of a customer based on sensor data from said one or more sensors to determine behavioral information, i.e., customer behavior determinator which interprets the video of the surveillance area of a second camera sensor to output a determination result of the customer's behavior captured by the second camera);
"a processor which specifies the customer's behavior as an abnormal behavior and a normal behavior based on the identification result of the product captured by the first camera and the determination result of the customer's behavior captured by the second camera, and generates a control signal depending on the specified result"; (Moller, Paras. 85-87, teaches the behavior system using behavioral information to detect if a customer is using the checkout counter the right way wherein the system will not prompt or guide the customer further or if the customer is acting in a deviant way wherein an output means may transmit a signal to the checkout system in the store or transmit instructions to execute an anti-theft operation, i.e., a processor that specifies the customer's behavior as normal being the correct way of operation and abnormal as deviant in which a control signal is generated based on the result);
"wherein the customer behavior determinator comprises an artificial neural network which analyzes the video of the surveillance area obtained by the second camera to determine the customer's behavior captured by the second camera"; (Moller, Paras. 22 and 45, teaches using convolutional neural networks to process the data gathered from the at least one sensor or plurality of sensors to determine behavioral information, i.e., customer behavior determinator comprises an artificial neural network to analyze the video of the area to determine customer's behavior);
"and wherein the artificial neural network is trained by learning a video of the customer's behavior captured by the second camera as the product is identified by the product identifier and a video of the customer's behavior captured by the second camera as the product is not identified by the product identifier to determine the customer's behavior that does not correspond to the behavior of locating a product in the product identification zone to be identified"; (Moller, Paras. 77-78, 91 and 109, teaches behavioral information being used to determine and output statistical data which is used to learn and update the system by using the statistical data as an input when determining future behavioral information wherein behavioral information includes determining the probability level related to the likelihood a customer will steal based on a failure to scan one or more articles, i.e., neural network is trained using video of customer behavior captured by the camera sensor as the product is identified by the product identifier and using video of customer behavior when the product is not identified by the product identifier to determine customer behavior that does not correspond to the behavior of locating a product in the product identification zone to be identified that is the likelihood of the customer stealing or leaving un-scanned items as the abnormal behavior).
The proposed combination as well as the motivation for combining the Moller, Johson, Gabale and Grigg references presented in the rejection of Claim 2, applies to claim 22.
However, the combination of references of Moller in view of Johnson, Gabale, and Grigg does not explicitly teach "a first camera (16) and a second camera (18) located within the area where the product identification zone (11-1) is located, in a top view".
In an analogous field of endeavor, Crooks teaches "a first camera (16) and a second camera (18) located within the area where the product identification zone (11-1) is located, in a top view"; (Crooks, Figs. 2-3 and Paras. 19 and 46-48, teaches both cameras may be placed above the scanner such as to provide overhead images wherein these cameras may be utilized to capture images of many things including items presented for scanning or weighing, an area around the terminal, a cashier at the terminal or a neighboring terminal, customers at the terminal or in line, and other areas, i.e., a first and second camera located within the area where the product identification zone is located in a top view).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Moller, Johnson, Gabale, and Grigg by including the first and second camera location within the area where the product identification zone is located in a top view taught by Crooks. One of ordinary skill in the art would be motivated to combine the references since it helps achieve a desired exposure (Crooks, Abstract, teaches the motivation of combination to be to help achieve a desired exposure within the captured image).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDREW STEVEN BUDISALICH whose telephone number is (703)756-5568. The examiner can normally be reached Monday - Friday 8:30am-5:00pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached on (571) 272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANDREW S BUDISALICH/Examiner, Art Unit 2662
/AMANDEEP SAINI/Supervisory Patent Examiner, Art Unit 2662
