DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This office action is in responsive to communication(s): original application filed on 09/14/2022, said application claims a priority filing date of 03/17/2020. Claims 1-12 and 14-21 are pending. Claims 1-2 and 12 are independent.
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(4) because (1) reference characters "106" in Page 23, line 30; Page 24, line 5; and Page 26, line 18 and "406" in FIG. 4 and Page 17, lines 5, 9, 12-13, and 23 have both been used to designate "capture input logic"; (2) reference characters "108" in Page 24, line 10 and "408" in FIG. 4; Page 17, lines 24-25 and 29; and Page 26, lines 22, 25-26, and 29 have both been used to designate "capture classifier (logic)"; and (3) reference characters "116" in Page 25, lines 21 and 26 and "416" in FIG. 4; Page 18, lines 3-5 and 17; and Page 19, line 11 have both been used to designate "heuristic logic". Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(4) because (1) reference character “106” has been used to designate both "CAPTURE DATUM" in FIG. 1 and Page 10, lines 1-2 and "capture input logic" in Page 23, line 30; Page 24, line 5; and Page 26, line 18; (2) reference character “108” has been used to designate both "CAPTURE CHARACTERISTIC" in FIG. 1 and Page 10, lines 5-6 and "capture classifier" in Page 24, line 10; and (3) reference character “116” has been used to designate both "END" step in FIG. 1 and Page 10, line 10 and "heuristic logic" in Page 25, lines 21 and 26. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they include the following reference character(s) not mentioned in the description: (1) 612, 614, 616, 618, and 622 in FIG. 6A; and (2) 624, 626, 634, and 636 in FIG. 6B. Corrected drawing sheets in compliance with 37 CFR 1.121(d), or amendment to the specification to add the reference character(s) in the description in compliance with 37 CFR 1.121(b) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
The drawings are objected to because (1) "FIG. 3B" appears to be "FIG. 6B" in FIG. 6A; and (2) "FIG. 3A" appears to be "FIG. 6A" in FIG. 6B because (a) there are no FIG. 3A and FIG. 3B; and (b) see Page 37, lines 5-14. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they do not include the following reference sign(s) mentioned in the description: "334" in Page 38, lines 16-18. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
The disclosure is objected to because of the following informalities:
in Abstract, line 8; and Page 3, lines 11 and 25, "... store the or each datum with ..." appears that some word(s) are missing between "the" and "or"; therefore, "... store each datum with ..." is considered;
in Page 5, line 11, "… images show in various aspects a specific product package" appears to be "… images show in various aspects of a specific product package";
in Page 9, lines 21-24, "The present technology thus provides automated annotation (tagging) of captured data with context metadata in real time or near real time to allow automated inputs to AI model learning or inference scenarios), e.g. in reinforcement learning, hybrid learning and other active learning)." appears to be "The present technology thus provides automated annotation (tagging) of captured data with context metadata in real time or near real time to allow automated inputs to AI model learning or inference scenarios (e.g. in reinforcement learning, hybrid learning and other active learning).";
in Page 23, line 30, "… the capture input logic 106 of …" appears to be "… the capture input logic 406 of …";
in Page 24, line 5, "The model used by capture input logic 106 will…" appears to be "The model used by capture input logic 406 will …";
in Page 24, line 10, "The model used by capture classifier 108 is …" appears to be "The model used by capture classifier 408 is …";
in Page 25, line 21, "… implementation of the heuristic logic 116 is …" appears to be "… implementation of the heuristic logic 416 is …";
in Page 25, line 26, "The heuristic logic 116 which operates …" appears to be "The heuristic logic 416 which operates …";
in Page 26, line 18, "… the capture input logic 106 detects …" appears to be "… the capture input logic 406 detects …";
in Page 28, lines 27-28, "… provide ML infrastructure for a for a retail checkout loss awareness application …" appears to be "… provide ML infrastructure for a retail checkout loss awareness application …";
in Page 38, line 18, "… (334 of Figure 6B) …" appears to be "… (634 of Figure 6B) …".
Appropriate correction is required.
Claim Objections
Claims 1-2, 9, 12, and 20 are objected to because of the following informalities:
in Claim 1, line 10; Claim 2, line 8; and Claim 12, line 9, "… store/storing the or each said datum with …" appears to be "… store/storing each said datum with …" (see also Specification Objection);
in Claim 1, lines 3-4 and Claim 2, lines 3-4, "… data capture logic operable to capture from an object at least one datum for inclusion in said dataset …" appears to be "… data capture logic operable to capture at least one datum of an object for inclusion in said dataset …" according to examples provided in Page 10, lines 2-4 ("… visual image data relating to characteristic forms and dimensions may be captured by a camera from an object placed in a capture area …"); Page 10, line 31 - Page 11, line 4 ("… a vehicle may enter a camera capture zone and be identified as a vehicle of the class "truck"; simultaneously, its registration plate may be captured …"); and Page 11, lines 8-9 ("… a set of images of a retail product may be captured by a camera at a point of sale …") of specification, and prevent confusion with examples provided in Page 2, lines 27-28 ("… the image data captured from the camera can be checked against the product identification data captured from the barcode reader …") of the specification.
in Claim 12, lines 3-4, "… capturing, by data capture logic, from an object at least one datum for inclusion in said dataset …" appears to be "… capturing, by data capture logic, at least one datum of an object for inclusion in said dataset …" for the same reason described above;
in Claim 9, lines 2-3, "… operable after training to detect a discrepancy between a current input and a stored said datum with an associated said annotation" appears to be "… said detector logic operable, after training, to detect the discrepancy between the current input and the stored said datum with the associated said annotation";
in Claim 20, lines 1-3, "… after training, detecting a discrepancy between a current input and a stored said datum with an associated said annotation" appears to be "… after training, detecting the discrepancy between the current input and the stored said datum with the associated said annotation".
Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: "signal component" in Claim 1.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1, 3-12, and 14-21 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 1 and 12 recites the limitation "… " in lines 2-1, which rendering the claim indefinite because it is unclear which model (first or second) is referred by ".
Claim limitation “signal component” in Claim 1 invokes 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. However, the written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed function and to clearly link the structure, material, or acts to the function. There is no corresponding structure described in the specification as performing the claimed function. Therefore, the claim is indefinite and is rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph.
Applicant may:
(a) Amend the claim so that the claim limitation will no longer be interpreted as a limitation under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph;
(b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(c) Amend the written description of the specification such that it clearly links the structure, material, or acts disclosed therein to the function recited in the claim, without introducing any new matter (35 U.S.C. 132(a)).
If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the function so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function, applicant should clarify the record by either:
(a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed function and clearly links or associates the structure, material, or acts to the claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed function. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181.
Claims 3-11 and 14-21 are rejected for fully incorporating the deficiency of their respective base claims.
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
The following is a quotation of pre-AIA 35 U.S.C. 112, fourth paragraph:
Subject to the following paragraph [i.e., the fifth paragraph of pre-AIA 35 U.S.C. 112], a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
Claims 9 and 20 are rejected under 35 U.S.C. 112(d) or pre-AIA 35 U.S.C. 112, 4th paragraph, as being of improper dependent form for failing to further limit the subject matter of the claim upon which it depends, or for failing to include all the limitations of the claim upon which it depends.
Claims 9 and 20 only recite "after training, detecting a discrepancy between a current input and a stored said datum with an associated said annotation" in lines 2-3 and 1-3 respectively, and this limitation has been recited in their respective based claims (see Claim 1, lines 13-14 and Claim 12, lines 11-12). Applicant may cancel the claim(s), amend the claim(s) to place the claim(s) in proper dependent form, rewrite the claim(s) in independent form, or present a sufficient showing that the dependent claim(s) complies with the statutory requirements.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-11 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter because "A machine learning apparatus comprising: a dataset for ...; data capture logic operable to ...; association logic operable to ...; annotator logic operable ... to ...; storage logic operable to ...; input logic to ... (for Claims 1-11); detector logic operable ... to ...; a signal component operable ... to ... (for Claims 1 and 3-11)" is cited without reciting any structure(s) of "the machine learning apparatus", and "the machine learning apparatus" includes "dataset" and various "logic(s)" which can be considered as "software" component(s). Also, see 112(f) claim interpolation and 112(b) rejection, the structure of "signal component" for performing claimed function is not described in the specification; and thus, "signal component" can also be considered as a "logic/software" component as well. Therefore, "dataset" and various "logic/software" components are claimed in Claims 1-11, and they are not one of the four categories of patent eligible subject matter.
Claims 1-12 and 14-21 are rejected under 35 U.S.C. 101 because the claimed invention is directed to abstract idea without significantly more. The claim(s) recite(s) "inferencing over a … model" (Claims 1, 3-11, and 14-21), "association logic operable to derive an additional characteristic of said object (corresponding to said at least one datum" (Claims 1-11), "deriving, by association logic, an additional characteristic of said object corresponding to said at least one datum" (Claims 12 and 14-21), "annotator logic operable in response to said data capture logic and said association logic to create an annotation linking said additional characteristic with said at least one datum (according to a second model)" (Claims 1-11), "responsive to said capturing and deriving, creating an annotation linking said additional characteristic with said at least one datum according to a second model" (Claims 12 and 14-21), "detector logic operable, after training said model with said dataset, to detect a discrepancy between a current input and a stored said datum with an associated said annotation (Claims 1 and 3-11), the discrepancy comprising a discrepancy in a retail product checkout process (Claim 11)", "detecting, after training said model with said dataset, a discrepancy between a current input and a stored said datum with an associated said annotation" (Claims 12 and 14-21), "detect/detecting a data pattern indicative of a datum class to derive at least one said additional characteristic associated with said datum" (Claims 3 and 14), "look/looking up a data record to derive at least one said additional characteristic associated with said datum" (Claims 4 and 15), "process/processing sound data (Claims 5 and 16), the sound data comprising voice data (Claims 6 and 17)", and "process/processing visual data (Claims 7 and 18), said visual data comprising at least one of a universal product code, a barcode, a QR code, a verbal label, a numeric label, a vehicle registration, an image mark, or a logotype (Claims 8 and 19)" which can be reasonably considered as involving mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper") or mathematical concepts/algorithms/calculations. This judicial exception is not integrated into a practical application because the claim recites additional elements/limitations "(providing) a dataset for input to a training procedure of a first machine learning model" (Claims 1-12 and 14-21), "data capture logic operable to capture from an object at least one datum for inclusion in said dataset" (Claims 1-11), "capturing, by data capture logic, from an object at least one datum for inclusion in said dataset" (Claims 12 and 14-21), "storage logic operable to store the or each said datum with an associated said annotation in said dataset" (Claims 1-11), "storing the or each said datum with an associated said annotation in said dataset" (Claims 12 and 14-21), "input logic to supply said dataset as machine learning input" (Claims 1-11), "supplying said dataset as machine learning input" (Claims 12 and 14-21), "a signal component, operable in response to said detecting said discrepancy, to emit an alert signal" (Claims 1 and 3-1), "emitting an alert signal in response to said detecting said discrepancy" (Claims 12 and 14-21) and "raise/raising an operator alert responsive to detecting said discrepancy" (Claims 10 and 21) which only amount to "apply it" with the use of generic computer components or insignificant extra solution activity. None of the additional limitations, taken alone or in combination, integrate the abstract idea into a practical application. The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because (a) t.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-4, 7-12, 14-15, and 18-21 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Bacelis et al. (US 2019/0236363 A1, pub. date: 08/01/2019), hereinafter Bacelis .
Independent Claims 1 and 2
Bacelis discloses a machine learning apparatus (Bacelis, ¶¶ [0032] and [0034] with 18 in FIG. 1: the environment 10 may include a graphical user interface (GUI) processing apparatus 12, a data repository 14, a machine learning (ML) training apparatus 18, a neural network 20, a sensing device 22, and a scanning apparatus 24; the server 18 may include one or more different computer processors at different locations but each connected to the network 16) comprising:
a dataset for input to a training procedure of a first machine learning model (Bacelis, ¶¶ [0004], [0009]-[0012], and [0015]-[0019]: modify the captured images in preparation for training an artificial intelligence apparatus to identify the items in the images; and a machine language (ML) model processor that determines whether the images training the artificial intelligence apparatus are correctly identified with machine-readable labels associated with the items; compare the machine-readable label affixed to the item and a valid image of the item to train a neural network; ¶¶ [0038]-[0043] with FIG. 1 and 110, 112, 114, and 116 in FIG. 2: record video clips of a checkout location in a digital format to a storage device 108, wherein the video clips can be partitioned into still images and placed into an image repository accessible for training the neural network 20 and auditing the images; at block 110, the digital images are shown as stored prior to evaluation/analysis, e.g., either for training or for artificial intelligence processing; at block 112, a training process by the neural network 20 and/or other artificial intelligence system may be performed; the image is evaluated and notated at GUI processing apparatus 12, and selected as being ingested into an artificial intelligence system for training and/or enhancement purposes; train an artificial neural network 20, which is applied at block 112 to identify the images by processing learning material from the image data; the results, e.g., cropped images, generated by the GUI processing apparatus 12 are output to a machine learning (ML) apparatus at block 114, where the training data is ingested by a machine learning apparatus, which in turn may be incorporated by the neural network 20 for identifying the item; a machine language (ML) model 116 (i.e., first model) may be implemented that identifies images of items with machine readable labels; the ML model 116 receives at another input data interrupted from the image and used that to weigh it against known data for the item to render a decision or score on the scan accuracy; ¶ [0044] with 320 and 330 in FIG. 4: the classified images 322A-C (generally, 322), also referred to as training images, are output to the neural network 20; ¶ [0045] with FIG. 7: the images 501D-501F are cropped from images 501A-501C respectively, and used to train the neural network 20 to identify any item of merchandise notwithstanding the presence of the scanning apparatus 24);
data capture logic operable to capture from an object at least one datum for inclusion in said dataset [by inferencing over a trained said first model] (NOTES; [] for Claim 1 only) (Bacelis, ¶¶ [0004]-[0008], [0010], [0012]-[0014], and [0017]: a first input for receiving a list of items with machine-readable labels; a second input for receiving a list of stores that have an inventory of the items in the list of items and that have at least one sensing device for capturing images of the items; a record for each of the list of stores includes a unique facility identifier and information about sensors available for generating images of items of interest of the list of items; the scan operation performed on the at least one machine-readable label; identify an image from a video feed taken of an item of interest at a store of the list of stores that is of interest with respect to confirming whether a machine-readable label is associated with a correct item; perform a scan operation performed on the at least one machine-readable label to distinguish the scanning apparatus from the item at which the at least one machine-readable label is located; an item of interest identified by contents of a machine-readable label affixed to the item; identify the image of the item to which the machine-readable label is affixed; ¶¶ [0032], [0034]-[0041] with 22 and 24 in FIG. 1and FIGS. 2-3: store and process a known UPC list 102, which includes a list of items with machine readable labels, e.g., stock keeping units (SKUs); the list 102 may include items of interest at risk of theft, fraud, and so on, and can be collected using historical data, data mining techniques, and so on; store and process a store list 103 including a list of retail establishments that have one or more cameras 22 positioned over a checkout counter; the cameras 22 are preferably high quality video cameras, for example closed-circuit television (CCTV) video devices; other sensing devices such as stereoscopic cameras, infrared, or IR sensors may be used in addition to or as an alternative to a camera, in particular any device capable of providing item unique attributes, such as size, shape, color, temperature, mass, weight, and so on; identifying each sensing device available for collecting images or other data regarding an item of interest; the digital video recorders (DVRs) of the cameras 22, or related security monitoring system, records video clips of a checkout location in a digital format to a storage device 108; identify a particular photograph from a video feed taken at a particular store, which may be of interest with respect to confirming whether a barcode is associated with a correct item; an item is scanned at 2:30 p.m. at a given register; an image stored at block 110 may contain the snapshot image of the register scan taken at the scanning apparatus 24, indicating that activity that occurred at the identified register at 2:30 p.m.; a scanner bed of a register 203 or other sensor apparatuses may process images of the item, to capture relevant features such as various angles, stock product images, and so on to confirm the item's scannable label matches the actual item scanned at the register 203; ¶¶ [0045]-[0046] with FIGS. 5 and 1: a plurality of images 401A-F (generally, 401) may be captured and used to create a neural network; a register having a scanner 24 can provide information on what items were scanned at a particular time, and the camera 22 can provide images taken at that time around the scanner 24; a scanning apparatus 24 at a checkout register scans the UPC of an inexpensive toy 31; a camera 22 captures an image of the television set at the scanner at the same time that the UPC is scanned);
association logic operable to derive an additional characteristic of said object [corresponding to said at least one datum] (NOTES; [] for Claim 1 only) (Bacelis, ¶¶ [0004], [0007]-[0008], [0012]-[0013], and [0019]: a data repository that stores the captured images of the items and that updates the electronic records to include an association to the captured images; a time of the scan operation performed on the at least one machine-readable label, and an identification of a checkout register where the scan operation is performed; a machine-readable label is associated with a correct item; identify the image of the item to which the machine-readable label is affixed; an image generated at a day and time stated in the time stamp; ¶¶ [0034]-[0037], [0039], and [0041] with 102, 103, and 106 in FIG. 2: a list of items with machine readable labels, e.g., stock keeping units (SKUs) or barcode labels; the UPC list 102 may include records, fields, or other electronic data that includes associations to previous or similar items, e.g., items sold in a previous season, limited time promotions, and so on; any sensing device(s) capable of providing item unique attributes, such as size, shape, color, temperature, mass, weight, and so on; output a listing 106 or table, matrix, or the like that includes one or more stores that have an item of interest identified by the contents of a machine-readable label affixed to the item; the listing 106 may include a date/time stamp that identifies when the item of interest was scanned, identifies the particular store, includes a register location/number reference and includes an index value that may direct the system to a set of images taken at the identified store at the date/time stamp, e.g., a register (reg.) identifying the location and type of register used, e.g., a unique identification to determine the location of an item scan operation; a date/time stamp that identifies a particular photograph from a video feed taken at a particular store, which may be of interest with respect to confirming whether a barcode is associated with a correct item; process images of the item, to capture relevant features such as various angles, stock product images, and so on; one or more multiple attributes to add additional context to image to further refine and improve detection variables; attributes may include other contexts such as lighting, hand or unrelated object in area, image error, incorrect image retrieved, blurriness, and so on);
annotator logic operable in response to said data capture logic and said association logic to create an annotation linking said additional characteristic with said at least one datum [according to a second model] (NOTES; [] for Claim 1 only) (Bacelis, ¶¶ [0004] and [0015]: modify the captured images in preparation for training an artificial intelligence apparatus to identify the items in the images; ¶¶ [0040], [0042], [0044]-[0045] with 112 in FIG. 2 and FIGS. 4-5: at block 112, a training process by the neural network 20 and/or other artificial intelligence system may be performed; the image is evaluated and notated at GUI processing apparatus 12; the stored images in the repository at block 110 may be analyzed at block 112; train an artificial neural network 20, which is applied at block 112 (i.e., second model) to identify the images by processing learning material from the image data; the receipt (310) by the GUI processing apparatus 12 of set of images 312A-C (generally, 312); the GUI processing apparatus 12 includes a display for visually displaying the images 312, where a user and/or computer program (i.e., second model) may annotate or otherwise modify the images 312; classify (320) the images, e.g., by identifying the images; the images 312 are classified for each SKU in a store; the classified images 322A-C (generally, 322), also referred to as training images, are output to the neural network 20; relevant products labelled with a SKU; the image 401F can be tagged with a UPC and item description and/or other identifier of the object in the image 401F; annotate the item of interest, e.g., a box of waffles, relative to the regions around the item to train the neural network 20 with accuracy);
storage logic operable to store the or each said datum with an associated said annotation in said dataset (Bacelis, ¶¶ [0004-[0008], and [0012]-[0014]: a data repository that stores the captured images of the items and that updates the electronic records to include an association to the captured images; the output of the scan table processing device includes a table comprising a plurality of data records, which includes at least one of a store identification, a time of the scan operation performed on the at least one machine-readable label, and an identification of a checkout register where the scan operation is performed; the output of the scan table processor includes a time stamp that identifies an image from a video feed taken of an item of interest at a store of the list of stores that is of interest with respect to confirming whether a machine-readable label is associated with a correct item; the listing includes a time stamp that identifies when the item of interest was scanned, identifies a store of the plurality of stores, an identification of a register at the identified store where the item is scanned, and an index value that provides an electronic storage location of an image generated at a day and time stated in the time stamp; ¶¶ [0032] and [0034]-[0041] with 14 in FIG. 1 and 102, 103, 104, 106, 108, and 110 in FIG. 2: a data repository 14; store and process a known UPC list 102, which includes a list of items with machine readable labels, e.g., stock keeping units (SKUs) or the barcode labels; the list 102 may include items of interest at risk of theft, fraud, and so on, and can be collected using historical data, data mining techniques, and so on; the UPC list 102 may be stored electronically, e.g., at a database or the like that includes store-related data, such as inventory details and so on; the UPC list 102 may include records, fields, or other electronic data that includes associations to previous or similar items, for example, items sold in a previous season, limited time promotions, and so on; store and process a store list 103 including a list of retail establishments that have an inventory of the items in the list of items of the UPC list 102; the store list 103 may be generated from existing asset inventory sheets, user-identified CCTV systems, and/or other store servers or data repositories used by retail establishments; this information may be generated as a record for each of the list of stores and stored at the data repository 14; the scan table processor 104 is a computer hardware processor, and may include a memory device or otherwise communicate with a storage device such the data repository 14 or the like to store and retrieve data to generate the scan table 104 and/or results generated by the scan table processor 104; the scan table 106 is constructed and arranged into a plurality of rows and columns, where each row includes data regarding an item scanned at a checkout counter; each column includes data identifying a store having security cameras at its checkout counters; the scan table processor 104 can output a listing 106 or table, matrix, or the like that includes one or more stores that have an item of interest identified by the contents of a machine-readable label affixed to the item; the listing 106 may include a date/time stamp that identifies when the item of interest was scanned, identifies the particular store, includes a register location/number reference and includes an index value that may direct the system to a set of images taken at the identified store at the date/time stamp, e.g., a register (reg.) identifying the location and type of register used, e.g., a unique identification to determine the location of an item scan operation; records video clips of a checkout location in a digital format to a storage device 108; at block 110, the digital images are shown as stored prior to evaluation/analysis, e.g., either for training or for artificial intelligence processing; the images may be categorized by the scan table listing 106; the listing 106, or output of the scan table processor 104, can include a date/time stamp that identifies a particular photograph from a video feed taken at a particular store, which may be of interest with respect to confirming whether a barcode is associated with a correct item; an image stored at block 110 may contain the snapshot image of the register scan taken at the scanning apparatus 24, indicating that activity that occurred at the identified register at 2:30 p.m.; a data repository at which the listing 106 is stored serves as the database storing all the reference and relevant materials needed to triangulate which cameras/time combinations need to be collected from storage device 108 and loaded into the repository at block 110; the stored images in the repository at block 110 may be analyzed at block 112);
input logic to supply said dataset as machine learning input (Bacelis, ¶¶ [0004], [0009]-[0012], and [0015]-[0019]: modify the captured images in preparation for training an artificial intelligence apparatus to identify the items in the images; and a machine language (ML) model processor that determines whether the images training the artificial intelligence apparatus are correctly identified with machine-readable labels associated with the items; compare the machine-readable label affixed to the item and a valid image of the item to train a neural network; ¶¶ [0038]-[0043] with FIG. 1 and 110, 112, 114, and 116 in FIG. 2: record video clips of a checkout location in a digital format to a storage device 108, wherein the video clips can be partitioned into still images and placed into an image repository accessible for training the neural network 20 and auditing the images; at block 110, the digital images are shown as stored prior to evaluation/analysis, e.g., either for training or for artificial intelligence processing; at block 112, a training process by the neural network 20 and/or other artificial intelligence system may be performed; the image is evaluated and notated at GUI processing apparatus 12, and selected as being ingested into an artificial intelligence system for training and/or enhancement purposes; train an artificial neural network 20, which is applied at block 112 to identify the images by processing learning material from the image data; the results, e.g., cropped images, generated by the GUI processing apparatus 12 are output to a machine learning (ML) apparatus at block 114, where the training data is ingested by a machine learning apparatus, which in turn may be incorporated by the neural network 20 for identifying the item; a machine language (ML) model 116 (i.e., first model) may be implemented that identifies images of items with machine readable labels; the ML model 116 receives at another input data interrupted from the image and used that to weigh it against known data for the item to render a decision or score on the scan accuracy; ¶ [0044] with 320 and 330 in FIG. 4: the classified images 322A-C (generally, 322), also referred to as training images, are output to the neural network 20; ¶ [0045] with FIG. 7: the images 501D-501F are cropped from images 501A-501C respectively, and used to train the neural network 20 to identify any item of merchandise notwithstanding the presence of the scanning apparatus 24; ¶¶ [0028]-[0029] and [0047] with FIGS. 9A-B and 10: testing a trained neural network using a closed circuit television (CCTV) footage; testing a trained neural network after cropping an image taken by a CCTV apparatus);
[detector logic operable, after training said model with said dataset, to detect a discrepancy between a current input and a stored said datum with an associated said annotation] (NOTES; [] for Claim 1 only) (Bacelis, ¶¶ [0004], [0008], and [0018]-[0019]: determine whether the images training the artificial intelligence apparatus are correctly identified with machine-readable labels associated with the items; confirming whether a machine-readable label is associated with a correct item; a determination that the machine-readable label is associated with an incorrect item at which the at least one machine-readable label is located; comparing the machine-readable label affixed to the item and a valid image of the item to train a neural network; and identifying the image of the item to which the machine-readable label is affixed; ¶¶ [0033] and [0041]-[0043] with 116 and 118 in FIG. 2 and FIG. 3: determining whether a machine-readable label is associated with a correct item; process images of the item to confirm the item's scannable label matches the actual item scanned at the register 203; the item confirmation processor can perform an electronic analysis of the two images and provide an automatic representation according to an object recognition application or other image analysis software; an item confirmation may be provided to the trainer with one or more reference samples for comparison; an exception list/triggering event 118 is generated from a comparison of the two inputs at the ML model 116 that includes an alert or exception regarding an item to which a scanned machine-readable label is associated that is not recognized); and
[a signal component, operable in response to said detecting said discrepancy, to emit an alert signal] (NOTES; [] for Claim 1 only) (Bacelis, ¶¶ [0011] and [0018]: generate an event in response to a determination that the machine-readable label is associated with an incorrect item at which the at least one machine-readable label is located; ¶ [0043] with FIG. 1 and 118 in FIG. 2: an exception list/triggering event 118 is generated from a comparison of the two inputs at the ML model 116 that includes an alert or exception regarding an item to which a scanned machine-readable label is associated that is not recognized; this detection and trigger event will occur in near real time when not training; the alert or exception may be output via the network 16 or via a local wireless connection such as Bluetooth or the like to a personal computer, a visual and/or audio alarm at the checkout counter, suspension of the current checkout transaction until authorized personnel are able to respond, security personnel notification, and so on; ¶ [0046] with FIG. 8: when the UPC has been fraudulently removed from the toy 31 and placed on an expensive television set 32, the neural network 20 determines that the item (i.e., the television) scanned does not fit the parameters for the toy, whereby an alert is automatically generated).
Claims 3 and 14
Bacelis discloses all the elements as stated in Claims 1 and 12 respectively and further discloses to detect/detecting a data pattern indicative of a datum class to derive at least one said additional characteristic associated with said datum (Bacelis, ¶ [0035]: any sensing device(s) capable of providing item unique attributes, such as size, shape, color, temperature, mass, weight, and so on; ¶ [0041] with 112 in FIG. 2: the stored images in the repository at block 110 may be analyzed at block 112; process images of the item, to capture relevant features such as various angles, stock product images, and so on to confirm the item's scannable label matches the actual item scanned at the register 203; one or more multiple attributes to add additional context to image to further refine and improve detection variables, wherein the attributes may include other contexts such as lighting, hand or unrelated object in area, image error, incorrect image retrieved, blurriness, and so on; ¶ [0045]: a process is performed multiple times for each SKU; e.g., multiple images, angles, lighting backgr