Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on July 10, 2026, has been entered.
Status of Claims
Applicant filed an amendment on July 10, 2026. Claims 1-8, 10-17, and 19-20 were pending in the Application. Claims 1, 11, and 20 have been amended. No new claims have been canceled, with claims 9 and 18 remaining canceled. No new claims have been added. Claims 1, 11, and 20 are the independent claims, the remaining claims depend on claims 1, 11, and 20. Thus claims 1-8, 10-17, and 19-20 are currently pending. After careful and full consideration of Applicant arguments and amendments, the Examiner finds them to be moot and/or not persuasive.
Response to Arguments
In the context of 35 U.S.C. §101, Applicant filed a Declaration by Graham Beauregard under 37 C.F.R. §1.132. Mr. Beauregard explains the following:
The detection and identification of physical objects using sensor data was a recognized challenge in the technical field of sensor-based object identification, “Computing systems could not perceive the physical world directly. Instead, these systems used sensor data, such as image data from a camera or load data from a weight sensor. A system that relied on a single sensor had only a partial description of the object and could not distinguish between objects whose differences lay outside what that sensor captured. For example, a system that relied on a camera alone could not tell two visually similar items apart, because the camera's image data did not encode the physical difference that separated them”.
This problem arose in verification systems that tried to determine whether a received identifier matched a physical item placed in a shopping cart: "Relying on one sensor's data left those systems unable to detect a mismatch between the added item and the provided identifier whenever the difference between them lay outside what that sensor captured".
That some verification systems combined vision and weight data, but these systems reduced each item's weight reading to a single numerical value.
How the claimed invention addresses these problems: “The claimed invention provides a particular solution to the problem of identifying an item from sensor data when a single load value cannot capture enough information about the item. The claimed invention captures a plurality of load measurements, each stored with a timestamp, that together form a load signal recorded by the load sensor over a period of time, and then uses a machine-learning model, trained as recited in the claims, to extract the information needed to verify the item from that load signal. ... By capturing a plurality of load measurements, each stored with its own timestamp, that together form a load signal measured over a period of time, the shopping cart preserves the information about the item that a single load value discards. The shape of the load signal over time, including how the load changes between measurements and what intermediate values the sensor records, remains available to the verification system rather than being collapsed into one number”.
That "the claimed invention improves the functionality of sensor-based object identification by capturing a load signal over a period of time and using a machine-learning model trained in the specific manner recited in the claims to extract from that load signal the information used to verify whether a received identifier matches the item placed in the shopping cart".
Applicant submits the following:
The claimed invention clearly meets the two-part test for reciting an improvement to a technical field, according to the MPEP and Ex parte Desjardins. Regarding the first step of this test, as explained, Mr. Beauregard identifies the disclosed method as an improvement to the problem that verification systems using a single load value could not distinguish items whose differences lay in information that a single value discarded. Accordingly, the disclosure provides "sufficient details such that one of ordinary skill in the art would recognize the claimed invention as providing an improvement."
Regarding the second part, the steps “capturing, …, a plurality of load measurements …”, “receiving, …, a load signal from the load sensor …”, and “inputting, …, the encoded feature vector to a machine-learning model …” confirm these are the ones that allow “the trained model ... [to] extract[] from the load signal over a period of time the information needed to verify the item, which a system that reads only a single load value had no way to obtain”.
The claims recite “encoding, …, a feature vector of the item …”, “inputting, …, the encoded feature vector to a machine-learning model …”, “accessing a plurality of training examples …”, and “for each of the training examples …” confirms that this training procedure "produces a model that has learned which features of a load signal over a period of time, taken together with image features and the received identifier, indicate whether the identifier matches the item".
The model trained this way "therefore extracts from the load signal over a period of time the information needed to verify the item, which a system that reads only a single load value had no way to obtain". Therefore, the claims "reflect the disclosed improvement in technology".
The claims meet the two-part test for reciting a technical improvement. Accordingly, the claims recite a practical application of a judicial exception and, as such, claims are patentable subject matter under Step 2A, Prong Two of the Alice test.
The Examiner acknowledges the Declaration by Graham Beauregard under 37 C.F.R. §1.132. Considering applicant’s arguments, in combination with the claim amendments, the Examiner hereby withdraws the rejection under 35 USC 101.
Claim Rejections - 35 USC § 101
Claims 1-8, 10-20 are considered subject matter eligible under 35 USC 101, for the following reasons. Representative claim 1 recites, the limitations:
accessing, by a processor of a shopping cart, an image of an item placed inside the shopping cart, wherein the image is captured by a camera coupled to the cart;
capturing, by the processor of the shopping cart, a plurality of load measurements for the item inside the shopping cart, wherein each load measurement is recorded by a load sensor coupled to a storage area of the shopping cart and is stored with a timestamp describing when the load measurement was recorded, wherein capturing the plurality of load measurements for the item inside the shopping cart comprises:
receiving, by the processor of the shopping cart, a load signal from the load sensor coupled to the storage area of the shopping cart, wherein the load signal comprises the plurality of load measurements measured by the load sensor over a period of time; encoding, by the processor of the shopping cart, a feature vector of the item based at least on the captured plurality of load measurements, the accessed image, and the received identifier;
inputting, by the processor of the shopping cart, the encoded feature vector to a machine- learning model that is trained to compute a confidence score, the confidence score describing a likelihood that the received identifier matches the item placed inside the shopping cart based on the captured load measurements and the accessed image, wherein the machine-learning model is trained by a process comprising:
accessing a plurality of training examples, wherein each training example comprises a training feature vector and a label, wherein the training feature vector describes a plurality of load measurements, an image, and an identifier, and where in the label indicates whether the identifier of the training example corresponds to the plurality of load measurements and the image of the training example;
for each of the training examples:
inputting the training feature vector of the training example to the machine-learning model to compute a confidence score for the training example;
comparing the confidence score of the training example to the label of the training example using a loss function; and
updating parameters of the machine-learning model based on the comparing;
determining, by the processor of the shopping cart, that the confidence score is less than a threshold confidence; and generating, by the processor of the shopping cart, a notification alerting an operator of an anomaly in the identifier based on the determination that the confidence score is less than the threshold confidence score.
The limitations above represent a technical solution to a technical problem. That is, the problem raised by a single sensor being insufficient to identify an item added to a cart. The claim solves this problem by gathering: a load from the load sensor coupled to a storage area of the shopping cart, an image captured by a camera coupled to the cart, and identifier. Encoding a feature vector with these three data points. Inputting the encoded feature vector into a machine learning model that is outputs a confidence score, trained using training feature vectors including load measurements, an image, and an identifier; and generating an alert based on the confidence score of the machine learning model. This solves the technical problem of misidentifying items based on images alone.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. § 102 and § 103 (or as subject to pre-AIA 35 U.S.C. § 102 and § 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. § 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U. S. 1. 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. § 103 are summarized as follows:
Determining the scope and contents of the prior art.
Ascertaining the differences between the prior art and the claims at issue.
Resolving the level of ordinary skill in the pertinent art.
Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-8, 11-17, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Sriraman et al (U. S. Patent Application Publication No. 20220157134 A1), herein referred to as Sriraman, in view of Hagen et al (U. S. Patent No. 12260386 B2), herein referred to as Hagen, in view of Gu et al (U. S. Patent No. 11373160 B2), herein referred to as Gu, and in further view of Khalili et al (U. S. Patent No. 11966901 B2), herein referred to as Khalili.
As per claims 1, 11, and 20, Sriraman discloses a method comprising: accessing, by a processor of a shopping cart (FIG. 2A, items 200, 220; para 41, “… FIG. 2A is a schematic diagram for an example of an automated physical shopping cart 200 for use in the disclosed technology that can detect changes in a weight of items in a basket 210 of cart 200, such as cart 120 of FIG. 1 … cart 200 includes a basket 210 for receiving items and a chassis 212 that supports cart 200 and provides mobility, such as with wheels 214A and 214B. A weight detector 224 is positioned between basket 210 and chassis 212 and can detect a weight of items in basket 210 …”; para 42, “… Cart 200 also includes cart controller 220, which is attached to cart 200 … and is in communication with optical device 222 for image recognition and weight detector 224 through a wired connection, such as a universal serial bus (USB) cable, or a wireless connection, such as a Bluetooth link. Other approaches to communicating data to cart controller 220 from optical device 222 and weight detector 224 can be utilized in accordance with the disclosed technology …”; FIG. 2B, items 220, 250; para 44, “… FIG. 2B is an architectural diagram showing an illustrative example of a client architecture for cart controller 220 that is suitable for application of the disclosed technology for automatically registering items in an automated physical shopping cart 200 … cart controller 220 includes an onboard computer 250 that can be capable of executing processes for optical image recognition, control, display, calculation and communication …”; para 45, “… Onboard computer 250 can communicate with optical devices 222, such as optical sensor 222A or camera 222B, and a variety of sensors 256 and Input/Output (I/O) devices through a local network or bus 251 …” ; FIG. 7, items 700, 702; para 103, “… FIG. 7, an illustrative computing device architecture 700 for a computing device that is capable of executing various software components is described herein for automatically registering items in an automated physical shopping cart. The computing device architecture 700 is applicable to computing devices, such as user client devices … the computing devices include, but are not limited to, mobile telephones, on-board computers, tablet devices, slate devices, portable video game devices, traditional desktop computers, portable computers (e.g., laptops, notebooks, ultra-portables, and netbooks), server computers, game consoles, and other computer systems. The computing device architecture 700 is applicable to the server device 110, client device 130 and controller device 220 shown in FIGS. 1, 2A and 2B, and computing devices 606A-N shown in FIG. 6 …”) an image of an item placed inside the shopping cart (para 35, “… determination is made from image data obtained from an optical device as to whether an item added to the cart can be identified from the image data from the optical device that correlates to the change in weight …”; para 36, “… One technical effect can be to improve the function of an automated cart by use an optical device to obtain item image data for items in the cart and using the item image data for item identification. Another technical effect can be to that image recognition for item identification can be performed in a user client device or a remote server instead of a cart controller …”), wherein the image is captured by a camera coupled to the cart (para 28, “… Optical devices, e.g. cameras, are frequently found in client devices, such as smart phones, tablets and laptop computers. Items may be scanned using an optical device and image recognition utilized to identify the item from image data from the optical device. The optical device can be an optical device attached to the cart or an optical device of a user client device. Image recognition can be performed, in various different examples, by an image recognition capability of a cart controller, an image recognition capability of the user client device, or a server or remote resource based image recognition capability …”; FIG. 2B, items 220, 222B; para 43, “…FIG. 2A, optical device 222 is configured on basket 210 to optically scan an item as it is added to the basket. As item 202 is placed in basket 210 at 204, optical device 222 collects image data for item 202 and provides the image data to cart controller 220 for image recognition processing … ”; FIG. 2B, items 250, 222A, 222B; para 45, “… Onboard computer 250 can communicate with optical devices 222, such as optical sensor 222A or camera 222B, and a variety of sensors 256 and Input/Output (I/O) devices through a local network or bus 251. Optical sensor 222A can capture video or photographic image data that can be processed to detect and identify objects, e.g. optical image recognition …”);
receiving, by the processor of the shopping cart (FIG. 2A, items 200, 220; paras 41, 42; FIG. 2B, items 220, 250; paras 44, 45; FIG. 7, items 700, 702; para 103), an identifier for the item placed inside the shopping cart (Sriraman, FIG. 4F, items 480, 482, 484; para 73, “… FIG. 4F is a control flow diagram illustrating an example of a process 480 in a client for communicating with a cart controller to update an inventory or shopping list …”; para 74, “… At 482, an item identifier is received from the cart controller, e.g. as a result of image recognition processing of item image data in the cart controller. At 484, the item identifier is checked against an inventory or shopping list, e.g. to determine if the item is included in an inventory in an inventory application or included in a set of item identifiers in a shopping list application. At 484, the item identifier can be processed for the inventory, e.g. add or subtract from inventory, or the item can be marked as satisfied in a shopping list, such as the X mark associated with ITEM ID A and ITEM ID D in the list in display area 318 of the GUI 310 illustrated in FIG. 3 …”); …
inputting, by the processor of the shopping cart (FIG. 2A, items 200, 220; paras 41, 42; FIG. 2B, items 220, 250; paras 44, 45; FIG. 7, items 700, 702; para 103), the encoded feature vector to a machine-learning model that is trained to compute a confidence score, the confidence score describing a likelihood that the received identifier matches the item placed inside the shopping cart and the accessed image (C/L 5/9-17, “… the correlated data can be generated based on applying a machine learning trained model to the weight change data and the motion data. The model was trained using a process that includes receiving training data for one or more other products that includes weight data, motion data, other sensor data, and positive product data correlations, training the model to correlate the received training data for the one or more other products, and outputting the model for runtime use …”; C/L 6/37-43, “…The disclosed techniques can also leverage machine learning techniques, such as machine learning trained models, to correlate different sensor signals ( e.g., weight, motion, etc.) and validate products that are added to the cart in real-time. This can provide for fast and accurate determinations about whether the guest is putting the expected product and quantity of the product in the cart …”; FIG. 2, items 204, 220A, 222B, 224C; C/L 13/28-42, “... the training in FIG. 2, the computing system 200 can receive training data 204 in step A (220). The training data 204 can be retrieved from a data store. The training data 204 can also be received from one or more other computing systems in communication with the computing system 200 via the network(s) 122. The training data 204 can include weight data 206, motion data 208, sensor data 210, and positive product data correlations 212. One or more other types of data can be included in the training data 204. The weight data 206 can be collected over time for a variety of products as they are put into shopping carts. The weight data 206 can measure fluctuations or changes (e.g., bumps) in weight in shopping carts when different products are placed, tossed, thrown, or otherwise put into the shopping carts …”; C/L 14/8-22, “… Training the model (step B, 222) can include inputting the training data 204 into the model and comparing output from the model to the positive product data correlations 212. Accuracy of the model can be determined based on this comparison and the model can be continuously trained and improved to refine its accuracy. Once the model is trained (and accuracy of the model is within a predetermined range or threshold level), the model can be outputted (step C, 224). Outputting the model can include storing the model in a data store for future retrieval and runtime use. Outputting the model can also include transmitting the model to one or more controllers 116 of shopping carts 100 and/or mobile devices 112 for local storage and fast deployment during runtime use …”) , wherein the machine-learning model is trained by a process comprising: …
In regards to claim 11, Sriraman further discloses a non-transitory computer-readable storage medium comprising stored instructions, which when executed by a processor of a shopping cart, cause the processor to (para 80, “… Some or all operations of the methods described herein, and/or substantially equivalent operations, can be performed by execution of computer-readable instructions included on a computer-storage media, as defined below. The term "computer-readable instructions," and variants thereof, as used in the description and claims, is used expansively herein to include routines, applications, application modules, program modules, programs, components, data structures, algorithms, and the like. Computer-readable instructions can be implemented on various system configurations, including single-processor or multiprocessor systems, minicomputers, mainframe computers, personal computers, hand-held computing devices, microprocessor-based, programmable consumer electronics, combinations thereof, and the like …”; FIG. 5, item 512; para 87, “… The mass storage device 512 and its associated computer-readable media provide non-volatile storage for the computer architecture 500. Although the description of computer-readable media contained herein refers to a mass storage device, such as a solid-state drive, a hard disk or CD-ROM drive, it should be appreciated by those skilled in the art that computer-readable media can be any available computer storage media or communication media that can be accessed by the computer architecture 500 …”) perform operations comprising: …
In regards to claim 20, Sriraman further discloses a shopping cart system (FIG. 1, items 100, 102, 110, 120, 130; para 40, “… FIG. 1 is an architectural diagram showing an illustrative example of an architecture 100 suitable for application of the disclosed technology for automatically registering items in an automated physical shopping cart. In the example of FIG. 1, automated shopping cart 120 can communicate with server 110 and mobile client 130 through network 102 …”) comprising: a shopping cart (FIG. 2A, item 200; para 41, “… FIG. 2A is a schematic diagram for an example of an automated physical shopping cart 200 for use in the disclosed technology that can detect changes in a weight of items in a basket 210 of cart 200, such as cart 120 of FIG. 1 … cart 200 includes a basket 210 for receiving items and a chassis 212 that supports cart 200 and provides mobility, such as with wheels 214A and 214B. A weight detector 224 is positioned between basket 210 and chassis 212 and can detect a weight of items in basket 210 ...”);
a load sensor coupled to a storage area of the shopping cart (FIG. 2A, item 224; para 41, “… FIG. 2A is a schematic diagram for an example of an automated physical shopping cart 200 for use in the disclosed technology that can detect changes in a weight of items in a basket 210 of cart 200, such as cart 120 of FIG. 1 … A weight detector 224 is positioned between basket 210 and chassis 212 and can detect a weight of items in basket 210 … when item 202 is placed in basket 210 at 204, weight detector 224 can detect the change … weight detector 224 can quantify the change in the weight of items in basket 210 …”);
a camera coupled to the shopping cart (FIG. 2A, items 200, 222; para 42, “… Cart 200 also includes cart controller 220, which is attached to cart 200 in this example and is in communication with optical device 222 for image recognition …”; FIG. 2B, items 222A, 22B; para 45, “… Onboard computer 250 can communicate with optical devices 222, such as optical sensor 222A or camera 222B, and a variety of sensors 256 and Input/Output (I/O) devices through a local network or bus 251. Optical sensor 222A can capture video or photographic image data that can be processed to detect and identify objects, e.g. optical image recognition …”; FIG. 7, items 700, 750; para 12-13);
a processor (FIG. 2A, item 220; para 42, “… Cart 200 also includes cart controller 220, which is attached to cart 200 in this example and is in communication with optical device 222 for image recognition and weight detector 224 through a wired connection, such as a universal serial bus (USB) cable, or a wireless connection, such as a Bluetooth link …”; FIG. 2B, items 220, 250; para 44, “… FIG. 2B is an architectural diagram showing an illustrative example of a client architecture for cart controller 220 that is suitable for application of the disclosed technology for automatically registering items in an automated physical shopping cart 200 … cart controller 220 includes an onboard computer 250 that can be capable of executing processes for optical image recognition, control, display, calculation and communication …”); and
a non-transitory computer-readable storage instructions that, when executed by a processor of a shopping cart system, cause the processor to perform operations comprising (para 80, “… Some or all operations of the methods described herein, and/or substantially equivalent operations, can be performed by execution of computer-readable instructions included on a computer-storage media, as defined below. The term "computer-readable instructions," and variants thereof, as used in the description and claims, is used expansively herein to include routines, applications, application modules, program modules, programs, components, data structures, algorithms, and the like. Computer-readable instructions can be implemented on various system configurations, including single-processor or multiprocessor systems, minicomputers, mainframe computers, personal computers, hand-held computing devices, microprocessor-based, programmable consumer electronics, combinations thereof, and the like …”; FIG. 5, item 512; para 87, “… The mass storage device 512 and its associated computer-readable media provide non-volatile storage for the computer architecture 500. Although the description of computer-readable media contained herein refers to a mass storage device, such as a solid-state drive, a hard disk or CD-ROM drive, it should be appreciated by those skilled in the art that computer-readable media can be any available computer storage media or communication media that can be accessed by the computer architecture 500 …”): …
Sriraman does not specifically disclose, however, Hagen discloses capturing, by the processor of the shopping cart (C/L 8/32-36, “… The shopping cart 100 can include an optional mounting fixture(s) 114 near a handle 102 of the cart 100, motion sensors 104A-N, weight sensor(s) 106A-N, an optional controller 116, an optional power source 118, and indicators 140A-B …”; FIG. 6, item 116; C/L 9/15-20, “… The optional controller 116 can be in communication (e.g., wired, wireless) with the mobile device 112, the motion sensor(s) 104A-N, and the weight sensor(s) 106A-N. The controller 116 can include one or more processors, CPU, RAM, and/or I/O. The controller 116 can perform one or more of the techniques described herein …”), a plurality of load measurements for the item inside the shopping cart, wherein the load measurement is recorded by a load sensor coupled to a storage area of the shopping cart (C/L 2/3-10, “… The disclosed technology can perform near instantaneous weight evaluations using weight sensor signals (and without having to wait for the sensor signals to settle out) by analyzing real-time changes in weight (e.g., weight bumps or fluctuations) over time with expected weight changes and/or other sensor data, such as motion of the products as they are added to (or removed from) the cart …”; C/L 13/61-14/7, “… for each product, the computing system 200 can map the weight data 206, motion data 208, and other sensor data 210 into n-dimensional (e.g., 2D, 3D, 4D, etc.) space. The computing system 200 can identify clusters of the mapped data and determine n-dimensional space values for each of the clusters. The clusters represent different data that can be correlated for the corresponding product and thus used to demonstrate, at a particular time, information about the product (e.g., a weight bump when the product is being thrown into a shopping cart at 2 m/s). The clusters can represent projected weight changes over time for the product, which, during runtime, can be compared to expected weight changes over time for the product to validate the product …”) ...
wherein capturing the plurality of load measurements for the item inside the shopping cart comprises: receiving, by the processor of the shopping cart (C/L 8/32-36; FIG. 6, item 116; C/L 9/15-20) a load signal from the load sensor coupled to the storage area of the shopping cart, wherein the load signal comprises the plurality of load measurements measured by the load sensor over a period of time (C/L 15/27-49, “… the weight data can include signatures of weight values detected in real-time in the cart. Weight in the cart can fluctuate over a period of time when a product is added to the cart and until the product settles down in the cart … when a milk carton is put in the cart, the detected weight can fluctuate significantly and potentially for a long period of time because it can take more time for the liquid contents of the milk carton to settle (e.g., the milk in the carton may continue to slosh around for some time, which can cause bumps in the detected weight in the cart). In comparison, if a hollow box is put in the cart, the detected weight can fluctuate less because the weight of the hollow, solid box can settle quickly. Thus, there can be fewer bumps in weight over time for the hollow box than for the milk carton. Moreover, a speed at which the product is put into the cart can also impact the weight values that are detected in real-time. If … the milk carton is thrown into the cart, then the weight data may demonstrate greater fluctuations (e.g., bumps) in weight over a longer period of time than when the milk carton is slowly placed inside the cart. Thus, a combination of weight and motion data can be beneficial to determine whether the product being added to the cart is the same product that was scanned …”);
encoding, by the processor of the shopping cart (C/L 8/32-36; FIG. 6, item 116; C/L 9/15-20), a feature vector of the item based at least on the captured plurality of load measurements (C/L 5/9-17, “… the correlated data can be generated based on applying a machine learning trained model to the weight change data and the motion data. The model was trained using a process that includes receiving training data for one or more other products that includes weight data, motion data, other sensor data, and positive product data correlations, training the model to correlate the received training data for the one or more other products, and outputting the model for runtime use …”; C/L 6/37-43, “… The disclosed techniques can also leverage machine learning techniques, such as machine learning trained models, to correlate different sensor signals (e.g., weight, motion, etc.) and validate products that are added to the cart in real-time. This can provide for fast and accurate determinations about whether the guest is putting the expected product and quantity of the product in the cart …”; FIG. 2, items 204, 206, 212, 222; C/L 13/28-42, “… the training in FIG. 2, the computing system 200 can receive training data 204 in step A (220). The training data 204 can be retrieved from a data store. The training data 204 can also be received from one or more other computing systems in communication with the computing system 200 via the network(s) 122. The training data 204 can include weight data 206, motion data 208, sensor data 210, and positive product data corre-lations 212. One or more other types of data can be included in the training data 204. The weight data 206 can be collected over time for a variety of products as they are put into shopping carts. The weight data 206 can measure fluctuations or changes (e.g., bumps) in weight in shopping carts when different products are placed, tossed, thrown, or otherwise put into the shopping carts), the accessed image, and the received identifier …”), the accessed image (C/L 13/47-54, “… The sensor data 210 can also be collected over time for a variety of products as they are put into shopping carts. The sensor data 210 can include information such as RFID scans, image data, and/or shadow maps of light obstructed by a product as it enters a shopping cart. The positive product data correlations 212 can be actual, verified correlations of different data for the variety of products …”), and the received identifier (FIG. 4, items 400, 402, 404, 406, 408; C/L 17/38-51, “… the process 400 in FIG. 4, a product scan can be performed at the mobile device 112 (402). Scanning the mobile device can include taking a picture or capturing other image data of the product using an image sensor of the mobile device. The image data can depict a portion of the product having a unique identifier, such as a barcode, SKU, UPC, etc. In 404, the mobile device 112 can retrieve product information using the unique identifier that was captured in the product scan … the controller 116 can retrieve the product information instead of the mobile device 112. The mobile device 112 can transmit the product information to the controller 116 (406), which the controller 116 receives in 408 …”); …
accessing a plurality of training examples, wherein each training example comprises a training feature vector and a label, wherein the training feature vector describes a plurality of load measurements, an image, and an identifier, and where in the label indicates whether the identifier of the training (FIG. 2, items 204, 222B; C/L 14/8-13, “… Training the model (step B, 222) can include inputting the training data 204 into the model and comparing output from the model to the positive product data correlations 212. Accuracy of the model can be determined based on this comparison and the model can be continuously trained and improved to refine its accuracy …”) example corresponds to the plurality of load measurements (FIG. 5B, item 520; C/L 21/66-22/1, “… FIG. 5B is a graphical depiction 520 of real-time weight signatures for products that are added to a shopping cart. The graph 520 represents curves for three products: A, B, and C …”; C/L 22/5-14, “… The three product weight signatures shown in FIG. 5B may not all be detected/received at the same time … when weight change is detected in the cart, the controller may only receive the detected weight signature for a most recent product that was scanned by the user at the user's mobile device. As shown in the graph 520, the curves for each of the products A, B, and C can fluctuate depending on their respective weights, contents, and speeds at which they enter or entered the cart …”; FIG. 5C, items 530, 540; C/L 22/38-51, “… FIG. 5C illustrate graphical depictions 530 and 540 comparing the expected weight signatures with actual weight signatures for the products … the graph 530 depicts the detected weight changes over time for product A against the expected weight signatures for the scanned product A. The graph 540 depicts the detected weight changes over time for product B against the expected weight signatures for the scanned product B. Once the detected weight changes for the product A are scaled and normalized to match scaling of the expected weight signature for the scanned product in the graph 530, it can be determined (e.g., by the controller 116 and/or the mobile device 112) that the product A matches the scanned product … the product is validated …”) and the image (FIG. 4, items 402, 404, 406, 408; C/L 17/38-50, “… the process 400 in FIG. 4, a product scan can be performed at the mobile device 112 (402). Scanning the mobile device can include taking a picture or capturing other image data of the product using an image sensor of the mobile device. The image data can depict a portion of the product having a unique identifier, such as a barcode, SKU, UPC, etc. In 404, the mobile device 112 can retrieve product information using the unique identifier that was captured in the product scan… the controller 116 can retrieve the product information instead of the mobile device 112. The mobile device 112 can transmit the product information to the controller 116 (406), which the controller 116 receives in 408 …”) of the training example;
for each of the training examples: inputting the training feature vector of the training example to the machine-learning model to compute a confidence score for the training example (FIG. 2, items 204, 222B; C/L 14/8-13, “… Training the model (step B, 222) can include inputting the training data 204 into the model and comparing output from the model to the positive product data correlations 212. Accuracy of the model can be determined based on this comparison and the model can be continuously trained and improved to refine its accuracy …”); …
updating parameters of the machine-learning model based on the comparing (C/L 14/14-21, “… Once the model is trained (and accuracy of the model is within a predetermined range or threshold level), the model can be outputted (step C, 224). Outputting the model can include storing the model in a data store for future retrieval and runtime use. Outputting the model can also include transmitting the model to one or more controllers 116 of shopping carts 100 and/or mobile devices 112 for local storage and fast deployment during runtime use …”; C/L 20/23-46, “… The weight profiles can indicate different weights and/or characteristics of the cart based on what items are already in the cart … machine learning techniques, algorithms, and/or models can be used to leverage and correlate a variety of parameters about the current state of the cart and the item entering the cart to generate an accurate weight signature for the item entering the cart. The parameters that can be used for training a machine learning model and as input during runtime use of the model can include, for each of the items already in the cart, volume, quantity, fluid/solid characteristics, item weight, and other parameters … when the controller determines a fairly high confidence of a particular item that is added to the cart, the controller can train an existing model or a new model(s) with information about the particular item as well as the parameters about the items already in the cart to then determine a weight signature for the particular item … training can be performed in an iterative feedback loop in which data is collected about items as they enter carts and used in combination with known information about items already in the carts to improve accuracy of determining weight signatures when the carts may be filled with one or more different types of items …”);
determining, by the processor of the shopping cart (C/L 8/32-36; FIG. 6, item 116; C/L 9/15-20), that the confidence score is less than a threshold confidence (FIG. 4, items 422, 424, 426; C/L 18/19-35, “… Once the controller 116 receives the product information, weight change data, and product motion data, the controller 116 can correlate the data (422) … the controller 116 can apply one or more machine learning trained models to correlate the received information/ data. The controller 116 can determine whether the correlated data is within threshold range(s) of the product information in 424. If the correlated data is within the threshold range(s) of the product information, the controller 116 can validate the product … the product put in the cart is likely the same as the product that was scanned by the mobile device 112. Conversely, if the correlated data is not within the threshold range(s) of the product information, the controller 116 may not validate the product; the product put in the cart likely is not the same as the product that was scanned by the mobile device 112 …”); and
generating, by the processor of the shopping cart (C/L 8/32-36; FIG. 6, item 116; C/L 9/15-20), a notification alerting an operator of an anomaly in the identifier based on the determination that the confidence score is less than the threshold confidence score (C/L 18/43-54, “… The controller 116 can then generate a notification about the determination in 426 … the notification can be a message or other form of output presented to the user of the mobile device 112. The notification can indicate that the product is validated (based on the controller 116 determining in block 424 that the correlated data is within the threshold range(s) of the product information). The notification may also indicate that the product is not validated and that the user should take some action in order to correct the error they made, such as removing the product put in the cart and replacing it with the actual scanned product or scanning the product put in the cart …”).
Hagen discloses a shopping cart with weight bump validation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include a shopping cart with weight bump validation, as in Hagen, to improve and/or enhance the technology for an automated shopping cart, as in Sriraman, because it would amount to combining elements that in the combination would perform the same function as they functioned separately. One of ordinary skill in the art before the effective filing date of the invention would have been motivated to combine the references to validate aspects of items as they are added to or removed from a shopping cart performing near instantaneous weight evaluations using weight sensor signals (and without having to wait for the sensor signals to settle out) by analyzing real-time changes in weight over time with expected weight changes and/or other sensor data, such as motion of the products as they are added to (or removed from) the cart, thus automatically validating aspects of products (e.g., items).
Sriraman and Hagen do not specifically disclose, however, Gu discloses … and is stored with a timestamp describing when the load measurement was recorded (FIG. 8, item 840; Table 1, C/L 16/49-67, “… the computer system of the store may log the identified interactions at step 840 … each interaction may be added as an entry in a log associated with the person. Each entry in the log may include information of one or more identified interactions of the person with one or more product items … once the computer system determines that a customer has taken a bottle of drink, it may add an entry to the log associated with the customer to represent this interaction … the computer system may first obtain the log file associated with the customer and the product item information of the bottle of drink. The computer system may then add an entry as shown in Table 1 to the log … the information in an entry may be encoded or encrypted. The entry may comprise less or more fields to meet the needs of different use cases … the entry may log weight information of the item being removed …”), …
comparing the confidence score of the training example to the label of the training example (C/L 14/61-15/6, “… data from different sensors may indicate different identities of a product item. The computer system may determine one of the identities based on a confidence score associated with each identity … the computer system may determine, based on data received from a first sensor, a first identity for the product item associated with a first confidence score. The computer system may determine, based on data received from a second sensor, a second identity for the product item associated with a second confidence score. The computer system may select one of the first identity and the second identity as the identity of the product item based on a comparison of the first confidence score and the second confidence score …”) …
Gu discloses monitoring shopping activities using weight data in a store. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include monitoring shopping activities using weight data in a store, as in Gu; and to include a shopping cart with weight bump validation, as in Hagen, to improve and/or enhance the technology for an automated shopping cart, as in Sriraman, because it would amount to combining elements that in the combination would perform the same function as they functioned separately. One of ordinary skill in the art before the effective filing date of the invention would have been motivated to combine the references to monitor shopping activities using weight data in a store to facilitate anonymous checkout in a store, and to minimize and/or eliminate human or manual involvement in making product identification more accurate, and to reduce fraud and/or fraudulent activities.
Sriraman, Hagen, and Gu do not specifically disclose, however, Khalili discloses … using a loss function (C/L 7/19-23, “… The purpose of the learning of the artificial neural network may be to determine the model parameters that minimize a loss function. The loss function may be used as an index to determine optimal model parameters in the learning process of the artificial neural network …”; C/L 11/40-44, “… The objective of training an ANN is to determine a model parameter for significantly reducing a loss function. The loss function may be used as an indicator for determining an optimal model parameter in a learning process of an artificial neural network …”); and …
Khalili discloses automated shopping experience using cashier-less systems. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include automated shopping experience using cashier-less systems, as in Khalili; to include monitoring shopping activities using weight data in a store, as in Gu; and to include a shopping cart with weight bump validation, as in Hagen, to improve and/or enhance the technology for an automated shopping cart, as in Sriraman, because it would amount to combining elements that in the combination would perform the same function as they functioned separately. One of ordinary skill in the art before the effective filing date of the invention would have been motivated to combine the references to overcome the inaccuracies of automated shopping experiences due to limitations with camera and sensor technology or customers trying to fool the system, thus reducing fraud and/or fraudulent activities.
As per claims 2 and 12, Sriraman, Hagen, Gu, and Khalili disclose all the limitations of claims 1 and 11. Sriraman further discloses the method of claim 1, further comprising: determining, by the processor of the shopping cart (FIG. 2A, items 200, 220; paras 41, 42; FIG. 2B, items 220, 250; paras 44, 45; FIG. 7, items 700, 702; para 103), the identifier for the item based on one or more of: a user input selecting the identifier of the item, wherein the selection of the identifier is made via a graphical user interface on the shopping cart (FIG. 3A, items 300, 310, 312, 314, 316, 318; para 52, “… FIG. 3A is a schematic diagram for an example of a client 300 for use in providing a user interface for the disclosed technology for automatically registering items in an automated physical shopping cart … client 300 includes a graphical user interface (GUI) screen 310 that can display graphical data and information … display area 312 of GUI 310 is dedicated to display of an item identifier and information, such as a UPC number, name and price of an item. Display area 314 is dedicated to display of a total price of items in the cart. Display area 316 is dedicated to notifications to a user, such as a notification that an item has been weight detected, but an optical scan has failed. Display area 318 is dedicated to displaying a shopping list of items that includes an indication of whether an item is in the shopping cart. Additional display areas can be utilized for a variety of information, such as inventory data …”; FIG. 3A, item 300; para 54, “… client 300 can be implemented as a dedicated client device attached to a shopping cart and in communication with cart controller 220 … client 300 can be a client integrated with cart controller 220 … client 300 can be implemented as a client application residing on a user client device, such as a smart phone or a tablet …”); or …
Sriraman does not specifically disclose, however, Hagen discloses a machine-learning model trained to identify the item by matching the item to a candidate item of a set of candidate items (FIG. 2, items 222 (step B), 224 (step C); C/L 14/8-21, “… Training the model (step B, 222) can include inputting the training data 204 into the model and comparing output from the model to the positive product data correlations 212. Accuracy of the model can be determined based on this comparison and the model can be continuously trained and improved to refine its accuracy. Once the model is trained (and accuracy of the model is within a predetermined range or threshold level), the model can be outputted (step C, 224). Outputting the model can include storing the model in a data store for future retrieval and runtime use. Outputting the model can also include transmitting the model to one or more controllers 116 of shopping carts 100 and/or mobile devices 112 for local storage and fast deployment during runtime use …”.
Hagen discloses a shopping cart with weight bump validation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include a shopping cart with weight bump validation, as in Hagen; to include automated shopping experience using cashier-less systems, as in Khalili; to include monitoring shopping activities using weight data in a store, as in Gu, to improve and/or enhance the technology for an automated shopping cart, as in Sriraman, because it would amount to combining elements that in the combination would perform the same function as they functioned separately. One of ordinary skill in the art before the effective filing date of the invention would have been motivated to combine the references to validate aspects of items as they are added to or removed from a shopping cart performing near instantaneous weight evaluations using weight sensor signals (and without having to wait for the sensor signals to settle out) by analyzing real-time changes in weight over time with expected weight changes and/or other sensor data, such as motion of the products as they are added to (or removed from) the cart, using machine learning trained models to correlate the various sensor data, thus automatically validating aspects of products (e.g., items).
As per claims 3 and 13, Sriraman, Hagen, Gu, and Khalili disclose all the limitations of claims 1 and 11. Sriraman further discloses the method of claim 1, wherein determining the load measurement for the item comprises: … by the processor of the shopping cart (FIG. 2A, items 200, 220; paras 41, 42; FIG. 2B, items 220, 250; paras 44, 45; FIG. 7, items 700, 702; para 103), …
… by the processor of the shopping cart (FIG. 2A, items 200, 220; paras 41, 42; FIG. 2B, items 220, 250; paras 44, 45; FIG. 7, items 700, 702; para 103), …
… by the processor of the shopping cart (FIG. 2A, items 200, 220; paras 41, 42; FIG. 2B, items 220, 250; paras 44, 45; FIG. 7, items 700, 702; para 103), …
Sriraman does not specifically disclose, however, Gu discloses … determining, …, a timestamp describing when the identifier for the item was received (FIG. 8, item 840; Table 1, C/L 16/49-67, “… the computer system of the store may log the identified interactions at step 840 … each interaction may be added as an entry in a log associated with the person. Each entry in the log may include information of one or more identified interactions of the person with one or more product items … once the computer system determines that a customer has taken a bottle of drink, it may add an entry to the log associated with the customer to represent this interaction. Specifically, the computer system may first obtain the log file associated with the customer and the product item information of the bottle of drink. The computer system may then add an entry as shown in Table 1 to the log … the information in an entry may be encoded or encrypted. The entry may comprise less or more fields to meet the needs of different use cases … the entry may log weight information of the item being removed …”); …
identifying, …, a subset of the plurality of load measurements recorded by the load sensor within a threshold timeframe of the timestamp describing when the identifier was determined (FIG. 8, item 840; Table 1, C/L 16/49-67); and
identifying, …, a load measurement of the subset of load measurements recorded nearest to the timestamp describing when the identifier for the item was determined based on the timestamp for the identified load measurement (FIG. 8, item 840; Table 1, C/L 16/49-67).
Gu discloses monitoring shopping activities using weight data in a store. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include monitoring shopping activities using weight data in a store, as in Gu; to include automated shopping experience using cashier-less systems, as in Khalili; and to include a shopping cart with weight bump validation, as in Hagen, to improve and/or enhance the technology for an automated shopping cart, as in Sriraman, because it would amount to combining elements that in the combination would perform the same function as they functioned separately. One of ordinary skill in the art before the effective filing date of the invention would have been motivated to combine the references to monitor shopping activities using weight data in a store to facilitate anonymous checkout in a store, and to minimize and/or eliminate human or manual involvement in making product identification more accurate, and to reduce fraud and/or fraudulent activities.
As per claims 4 and 14, Sriraman, Hagen, Gu, and Khalili disclose all the limitations of claims 1, 3, 11, and 13. Sriraman further discloses the method of claim 3, wherein identifiers of multiple items were determined during the threshold timeframe, the method further comprising: generating, by the processor of the shopping cart (FIG. 2A, items 200, 220; paras 41, 42; FIG. 2B, items 220, 250; paras 44, 45; FIG. 7, items 700, 702; para 103), a queue of items identified during the threshold timeframe, wherein the queue of items are ordered sequentially (FIG. 3A, items 300, 318; para 52, “… FIG. 3A is a schematic diagram for an example of a client 300 for use in providing a user interface for the disclosed technology for automatically registering items in an automated physical shopping cart … client 300 includes a graphical user interface (GUI) screen 310 that can display graphical data and information … Display area 318 is dedicated to displaying a shopping list of items that includes an indication of whether an item is in the shopping cart. Additional display areas can be utilized for a variety of information, such as inventory data …”; FIG. 4A, item 406; para 59, “… At 406, image recognition processing is performed on the received image data to determine whether a correlated item was identified that correlates to the detected change in weight … an item can be identified from image data received close in time to the time that the change in weight is detected … timestamp data from the image data can be used to determine whether an item was identified in the cart within a time range of the time that the change in weight was detected …”) …
identifying, by the processor of the shopping cart (FIG. 2A, items 200, 220; paras 41, 42; FIG. 2B, items 220, 250; paras 44, 45; FIG. 7, items 700, 702; para 103), a number of load measurements of the plurality of load measurements corresponding to a number of items in the queue (FIG. 4A, item 406; para 59, “… At 406, image recognition processing is performed on the received image data to determine whether a correlated item was identified that correlates to the detected change in weight … an item can be identified from image data received close in time to the time that the change in weight is detected … timestamp data from the image data can be used to determine whether an item was identified in the cart within a time range of the time that the change in weight was detected …”); and
assigning, by the processor of the shopping cart (FIG. 2A, items 200, 220; paras 41, 42; FIG. 2B, items 220, 250; paras 44, 45; FIG. 7, items 700, 702; para 103), each of the number of load measurements to an item in the queue sequentially (FIG. 3A, items 300, 318; para 52, “… FIG. 3A is a schematic diagram for an example of a client 300 for use in providing a user interface for the disclosed technology for automatically registering items in an automated physical shopping cart … client 300 includes a graphical user interface (GUI) screen 310 that can display graphical data and information … Display area 318 is dedicated to displaying a shopping list of items that includes an indication of whether an item is in the shopping cart. Additional display areas can be utilized for a variety of information, such as inventory data …”; FIG. 4A, item 406; para 59, “… At 406, image recognition processing is performed on the received image data to determine whether a correlated item was identified that correlates to the detected change in weight … an item can be identified from image data received close in time to the time that the change in weight is detected … timestamp data from the image data can be used to determine whether an item was identified in the cart within a time range of the time that the change in weight was detected …”) …
Sriraman does not specifically disclose, however, Gu discloses … based on timestamps when the identifier for each item in the queue was received (FIG. 8, item 840; Table 1, C/L 16/49-67); …
… based on the timestamp for the identified load measurement (FIG. 8, item 840; Table 1, C/L 16/49-67).
Gu discloses monitoring shopping activities using weight data in a store. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include monitoring shopping activities using weight data in a store, as in Gu; to include automated shopping experience using cashier-less systems, as in Khalili; and to include a shopping cart with weight bump validation, as in Hagen, to improve and/or enhance the technology for an automated shopping cart, as in Sriraman, because it would amount to combining elements that in the combination would perform the same function as they functioned separately. One of ordinary skill in the art before the effective filing date of the invention would have been motivated to combine the references to monitor shopping activities using weight data in a store to facilitate anonymous checkout in a store, and to minimize and/or eliminate human or manual involvement in making product identification more accurate, and to reduce fraud and/or fraudulent activities.
As per claim 5, Sriraman, Hagen, Gu, and Khalili disclose all the limitations of claim 1. Sriraman, Gu, and Khalili do not specifically disclose, however, Hagen discloses the method of claim 1, wherein capturing the plurality of load measurements for the item inside the shopping cart comprises identifying a plurality of load measurement recorded within a threshold timeframe of the identifier being received (FIG. 3, items 300, 304; C/L 14/22-28, “… FIG. 3 is a flowchart of a process 300 for using a machine learning model to validate a product … the model can be used to correlate, in real-time, different data signals received of a product being added to a shopping cart … the model can also be trained to compare the correlated data to expected data for the product …”; C/L 15/14-49, “… In 304, the controller can receive weight data in real-time … the controller can continuously poll weight sensor(s) of the cart at predetermined time intervals for changes in weight in the cart … once a product is scanned by the mobile device and the controller receives notification of the scan (such as receiving the product information in 302), the controller can poll the weight sensor(s) for changes in weight in the cart … the weight sensor(s) can detect changes in weight in the cart and automatically transmit those weight changes in real-time to the controller, without a request or poll from the controller … the weight data can include signatures of weight values detected in real-time in the cart. Weight in the cart can fluctuate over a period of time when a product is added to the cart and until the product settles down in the cart … when a milk carton is put in the cart, the detected weight can fluctuate significantly and potentially for a long period of time because it can take more time for the liquid contents of the milk carton to settle (e.g., the milk in the carton may continue to slosh around for some time, which can cause bumps in the detected weight in the cart). In comparison, if a hollow box is put in the cart, the detected weight can fluctuate less because the weight of the hollow, solid box can settle quickly. Thus, there can be fewer bumps in weight over time for the hollow box than for the 40 milk carton. Moreover, a speed at which the product is put into the cart can also impact the weight values that are detected in real-time … the milk carton is thrown into the cart, then the weight data may demonstrate greater fluctuations (e.g., bumps) in weight over a longer period of time than when the milk carton is slowly placed inside the cart. Thus, a combination of weight and motion data can be beneficial to determine whether the product being added to the cart is the same product that was scanned …”).
Hagen discloses a shopping cart with weight bump validation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include a shopping cart with weight bump validation, as in Hagen; to include automated shopping experience using cashier-less systems, as in Khalili; and to include monitoring shopping activities using weight data in a store, as in Gu, to improve and/or enhance the technology for an automated shopping cart, as in Sriraman, because it would amount to combining elements that in the combination would perform the same function as they functioned separately. One of ordinary skill in the art before the effective filing date of the invention would have been motivated to combine the references to validate aspects of items as they are added to or removed from a shopping cart performing near instantaneous weight evaluations using weight sensor signals (and without having to wait for the sensor signals to settle out) by analyzing real-time changes in weight over time with expected weight changes and/or other sensor data, such as motion of the products as they are added to (or removed from) the cart, thus automatically validating aspects of products (e.g., items).
As per claims 6 and 15, Sriraman, Hagen, Gu, and Khalili disclose all the limitations of claims 1 and 11. Sriraman, Gu, and Khalili do not specifically disclose, however, Hagen discloses the method of claim 1, wherein inputting the encoded feature vector to the machine-learning model to compute the confidence score further comprises:
determining, by the processor of the shopping cart (C/L 8/32-36; FIG. 6, item 116; C/L 9/15-20), a weight range for the item based on previously recorded load data encoded into the feature vector (FIG. 3, items 300, 310; C/L 16/13-20, “… The controller can then apply the correlation model to correlate the weight data and the sensor data (310). Correlating the weight data and the sensor data can include matching bumps (e.g., fluctuations) in detected weight with changes in position over time of the product as it is added to the cart … the correlated data can indicate when weight bumps occur based on a speed at which the product enters the cart …”); and
determining, by the processor of the shopping cart (C/L 8/32-36; FIG. 6, item 116; C/L 9/15-20), the confidence score based on whether the load measurement falls within the weight range (FIG. 3, items 300, 312; C/L 16/25-40, “… The controller can determine whether the model output is within a predetermined threshold range of the product information in 312 … the controller can compare a curve or other graphical depiction of the correlated data (e.g., weight change over time, based on speed of the product as the product enters the cart) to a curve or other graphical depiction of the expected weight data for the product. If the curve of the correlated data deviates from the curve of the expected weight data by more than a predetermined threshold amount, then the controller can determine that the product cannot be validated. If, on the other hand, the curve of the correlated data deviates from the curve of the expected weight data by less than the predetermined threshold amount or otherwise is similar or the same, then the controller can determine that the product in the cart matches the scanned product ...).
Hagen discloses a shopping cart with weight bump validation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include a shopping cart with weight bump validation, as in Hagen; to include automated shopping experience using cashier-less systems, as in Khalili; and to include monitoring shopping activities using weight data in a store, as in Gu, to improve and/or enhance the technology for an automated shopping cart, as in Sriraman, because it would amount to combining elements that in the combination would perform the same function as they functioned separately. One of ordinary skill in the art before the effective filing date of the invention would have been motivated to combine the references to validate aspects of items as they are added to or removed from a shopping cart performing near instantaneous weight evaluations using weight sensor signals (and without having to wait for the sensor signals to settle out) by analyzing real-time changes in weight over time with expected weight changes and/or other sensor data, such as motion of the products as they are added to (or removed from) the cart, using machine learning trained models to correlate the various sensor data, thus automatically validating aspects of products (e.g., items).
As per claims 7 and 16, Sriraman, Hagen, Gu, and Khalili disclose all the limitations of claims 1 and 11. Sriraman further discloses … the method further comprising: determining, by the processor of the shopping cart (FIG. 2A, items 200, 220; paras 41, 42; FIG. 2B, items 220, 250; paras 44, 45; FIG. 7, items 700, 702; para 103), the confidence score based on a comparison of the one or more visual features of the item extracted from the accessed image and one or more known visual features for the item associated with the identifier (FIG. 4A, items 400, 402, 404, 406, 410, 420; para 58, “… FIG. 4A is a control flow diagram showing an illustrative example of a process 400 for automatically registering items in an automated physical shopping cart in accordance with the disclosed technology. At 402, a change in weight of the items in a cart, such as cart 200, is detected, e.g. through a weight detection signal provided by a weight detector. At 404, image data is received from an optical device, e.g. optical device 222 …”; para 59, “… At 406, image recognition processing is performed on the received image data to determine whether a correlated item was identified that correlates to the detected change in weight … an item can be identified from image data received close in time to the time that the change in weight is detected … timestamp data from the image data can be used to determine whether an item was identified in the cart within a time range of the time that the change in weight was detected …”; para 60, “… If the item was successfully identified, then control branches at 410 to 420 … image data captured by optical device 222 can be provided to a remote server, which can identify the item through image recognition processing performed at the remote server … remote computing resources can be utilized to perform image recognition on the image data to identify items …”).
Sriraman, Gu, and Khalili do not specifically disclose, however, Hagen discloses the method of claim 1, wherein the confidence score is determined by inputting, by the processor of the shopping cart (C/L 8/32-36; FIG. 6, item 116; C/L 9/15-20), one or more visual features of the item extracted from the accessed image of the item placed in the shopping cart and one or more known visual features of the item associated with the identifier (FIG. 4, items 400, 402, 404, 406, 408; C/L 17/38-50, “… the process 400 in FIG. 4, a product scan can be performed at the mobile device 112 (402). Scanning the mobile device can include taking a picture or capturing other image data of the product using an image sensor of the mobile device. The image data can depict a portion of the product having a unique identifier, such as a barcode, SKU, UPC, etc. In 404, the mobile device 112 can retrieve product information using the unique identifier that was captured in the product scan … the controller 116 can retrieve the product information instead of the mobile device 112. The mobile device 112 can transmit the product information to the controller 116 (406), which the controller 116 receives in 408 …”), …
Hagen discloses a shopping cart with weight bump validation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include a shopping cart with weight bump validation, as in Hagen; to include automated shopping experience using cashier-less systems, as in Khalili; and to include monitoring shopping activities using weight data in a store, as in Gu, to improve and/or enhance the technology for an automated shopping cart, as in Sriraman, because it would amount to combining elements that in the combination would perform the same function as they functioned separately. One of ordinary skill in the art before the effective filing date of the invention would have been motivated to combine the references to validate aspects of items as they are added to or removed from a shopping cart performing near instantaneous weight evaluations using weight sensor signals (and without having to wait for the sensor signals to settle out) by analyzing real-time changes in weight over time with expected weight changes and/or other sensor data, such as motion of the products as they are added to (or removed from) the cart, using machine learning trained models to correlate the various sensor data, thus automatically validating aspects of products (e.g., items).
As per claims 8 and 17, Sriraman, Hagen, Gu, and Khalili disclose all the limitations of claims 1 and 11. Sriraman, Gu, and Khalili do not specifically disclose, however, Hagen discloses the method of claim 1, further comprising: determining, by the processor of the shopping cart (C/L 8/32-36; FIG. 6, item 116; C/L 9/15-20), an accepted variance in previously recorded load data for the item based on a distribution of the previously recorded load data (FIG. 5A, items 504, 506; C/L 20/60-21/10, “… the controller can receive the weight data in block 504 and then identify which category of products the product is most likely associated with based on that weight data … the controller can be loaded with a table or other type of data record indicating different product categories and their associated weight ranges. The controller can receive the weight data and compare the weight data to the weight ranges in the table to determine what category the product is likely associated with. The controller can then determine whether the weight data is within a threshold range for that category to then validate whether the product is the actual product that was scanned by the user using the shopping cart … the controller can also determine whether the weight data falls into a general range, threshold range(s), and/or clusters for expected behavior (e.g., weight changes) of products of a particular category and/or type to validate the product …”);
responsive to determining that the confidence score is less than a threshold confidence, comparing, by the processor of the shopping cart (C/L 8/32-36; FIG. 6, item 116; C/L 9/15-20), the plurality of load measurement for the item to the accepted variance (FIG. 5A, items 508, 510; C/L 21/27-49, “… In 508, the controller can compare the scaled weight data to the expected weight signature for the … the controller can graph the scaled weight change over time curve with the excepted weight signature curve. The controller can assess peaks and valleys in the curves and whether such peaks and valleys are close in amplitude, duration, and/or timing. Comparing the two curves can include using calculus or other computational functions to determine timing of inflection points between up and down portions (e.g., peaks and valleys) of the curves, when each curve reaches min and max values, and maximum amplitude values. The controller can compare the two curves based on looking at a particular period of time, such as between t=0 seconds and t=l seconds. As a result, the controller may not have to wait until more weight change data is collected and available to make a real-time determination of whether the product is validated … the controller can apply one or more machine learning models to compare the scaled weight data to the expected weight signature for the scanned product. The controller can determine whether the scaled weight signature is within a threshold range of the expected weight signature for the product (510) …”);
determining, by the processor of the shopping cart (C/L 8/32-36; FIG. 6, item 116; C/L 9/15-20), whether the plurality of load measurements are outside the accepted variance based on the comparing (FIG. 5A, item 512; C/L 21/50-56, “… the controller can determine how closely the detected weight changes over time aligns with the expected weight signature for the scanned product. The greater the deviation from the expected weight signature, the more likely the product added to the cart does not match the scanned product. The less deviation from the expected weight signature (e.g., deviation that is less than the threshold range), the more likely the product added to the cart matches the scanned product …”; C/L 21/62-64, “… If, on the other hand, the scaled weight data is not within the threshold range, the controller can generate output indicating that the product was not validated …”); and
responsive to determining the plurality of load measurement are outside the accepted variance, transmitting, by the processor of the shopping cart (C/L 8/32-36; FIG. 6, item 116; C/L 9/15-20), a notification to an operator identifying the anomaly as fraudulent (FIG. 4, items 424, 426, 428, 430, 432; C/L 18/19-35, “… Once the controller 116 receives the product information, weight change data, and product motion data, the controller 116 can correlate the data (422). As described herein (e.g., refer to FIGS. 2-3), the controller 116 can apply one or more machine learning trained models to correlate the received information/ data. The controller 116 can determine whether the correlated data is within threshold range(s) of the product information in 424. If the correlated data is within the threshold range(s) of the product information, the controller 116 can validate the product … the product put in the cart is likely the same as the product that was scanned by the mobile device 112. Conversely, if the correlated data is not within the threshold range(s) of the product information, the controller 116 may not validate the product; the product put in the cart likely is not the same as the product that was scanned by the mobile device 112 …”; C/L 18/43-54, “… The controller 116 can then generate a notification about the determination in 426 … the notification can be a message or other form of output presented to the user of the mobile device 112. The notification can indicate that the product is validated (based on the controller 116 determining in block 424 that the correlated data is within the threshold range(s) of the product information). The notification may also indicate that the product is not validated and that the user should take some action in order to correct the error they made, such as removing the product put in the cart and replacing it with the actual scanned product or scanning the product put in the cart …”; C/L 18/57-57, “… The controller 116 can transmit the notification in 428, which the mobile device 112 can receive in 430 … the controller 116 can transmit the notification or other notifications to other computing devices and/or systems, including but not limited to computing devices of retail environment employees. The mobile device 112 can output the notification in 432. The user can therefore be presented the notification. In some implementations, instead of or in addition to outputting the notification, the mobile device 112 can also perform some action in response to receiving the notification in 430 …”; FIG. 5A, item 514).
Hagen discloses a shopping cart with weight bump validation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include a shopping cart with weight bump validation, as in Hagen; to include automated shopping experience using cashier-less systems, as in Khalili; and to include monitoring shopping activities using weight data in a store, as in Gu, to improve and/or enhance the technology for an automated shopping cart, as in Sriraman, because it would amount to combining elements that in the combination would perform the same function as they functioned separately. One of ordinary skill in the art before the effective filing date of the invention would have been motivated to combine the references to validate aspects of items as they are added to or removed from a shopping cart performing near instantaneous weight evaluations using weight sensor signals (and without having to wait for the sensor signals to settle out) by analyzing real-time changes in weight over time with expected weight changes and/or other sensor data, such as motion of the products as they are added to (or removed from) the cart, thus automatically validating aspects of products (e.g., items).
Claims 10 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Sriraman et al (U. S. Patent Application Publication No. 20220157134 A1), herein referred to as Sriraman, in view of Hagen et al (U. S. Patent No. 12260386 B2), herein referred to as Hagen, in view of Gu et al (U. S. Patent No. 11373160 B2), herein referred to as Gu, in view of Khalili et al (U. S. Patent No. 11966901 B2), herein referred to as Khalili, and in further view of Bronicki et al (U. S. Patent No. 11423467 B2), herein referred to as Bronicki.
As per claims 10 and 19, Sriraman, Hagen, Gu, and Khalili disclose all the limitations of claims 1, 8, 11, and 17. Sriraman, Hagen, Gu, and Khalili do not specifically disclose, however, Bronicki discloses the method of claim 8, further comprising: responsive to determining the plurality of load measurements are outside the accepted variance, accessing, by the processor of the shopping cart (C/L 8/64-9/16, “… The at least one processor may be configured to receive image data captured in a retail store. A first shopping receptacle and a second shopping receptacle may be represented in the received image data. The at least one processor may also be configured to determine that the first shopping receptacle is associated with a first virtual shopping cart and that the second shopping receptacle is associated with a second virtual shopping cart different from the first virtual shopping cart, and to analyze the received image data to detect a shopper placing a first product in the first shopping receptacle and to detect the shopper placing a second product in the second shopping receptacle. The at least one processor may be further configured to, in response to detecting that the shopper placed the first product in the first shopping receptacle, automatically updating the first virtual shopping cart to include information associated with the first product, and in response to detecting that the shopper placed the second product in the second shopping receptacle, automatically updating the second virtual shopping cart to include information associated with the second product …”, purchase history for a user of the shopping cart, wherein the purchase history includes a number of fraudulent anomalies identified for the user (FIG. 28, items 2808, 2826; C/L 118/40-119/7, “… sensors communication module 2802 may receive an data from one or more sensors in retail store 105, captured data analysis module 2804 may use the received data to detect a shopper and to identify a plurality of product interaction events for the detected shopper, shopping data determination module 2806 may determine frictionless shopping data for the shopper, shoplift risk determination module 2808 may use database 2818 to determine a likelihood that the shopper will be involved in shoplifting, detail level determination module 2810 may determine the detail level of frictionless shopping data to provide to the shopper based on the determined shoplift risk, update rate determination module 2812 may determine update rate for updating the shopper with the shopping data also based on the determined shoplift risk, and shopper communication module 2814 may cause a delivery of the determined shopping data to the shopper …”; C/L 120/50-67, “… shoplift risk determination module 2808 may determine the likelihood that the shopper will be involved in shoplifting using one or a combination of the following modules: an action identification module 2820, a shopper recognition module 2822, a shopper trait estimation module 2824, and a shopping history determination module 2826. Modules 2820-2826 may be part of shoplift risk determination module 2808 or separate from shoplift risk determination module 2808 … the determined shoplift risk level may be an aggregation (e.g., a weighted combination) of two or more modules … shoplift risk determination module 2808 may make the determination of the shoplift risk level based on a weighted average shoplift risk level determined by at least some of modules 2820-2826. Different analyses may be assigned different weights to different modules, and the disclosed embodiments are not limited to any particular combination of analyses and weights …”; C/L 121/59-122/11, “… Shopping history determination module 2826 is configured retrieve a shopping history associated with a particular shopper from a shopping history database (e.g., part of databases 2818), and to determine the likelihood that the shopper will be involved in shoplifting based, at least in part, on the retrieved shopping history … the shopping history retrieved from the database may include previous questionable conduct by the shopper that may result in the determination of a higher risk of shoplifting … the shopping history database may store facial signatures of shoppers that previously visited retail store 105. The facial signatures may be used in identifying the shopper via the analysis of the image data and in retrieving the shopping history for the shopper from the shopping history database … the shopping history database may store a history records of returns made to the retail store by different shoppers. Thereafter, shopping history determination module 2826 may use the history of returns in determining the likelihood that a shopper will be involved in shoplifting …”);
determining, by the processor of the shopping cart (C/L 8/64-9/16), whether the number of fraudulent anomalies identified for the user exceeds a threshold (FIG. 28, item 2808, 2810, 2812; C/L 120/23-49, “… Shoplift risk determination module 2808 may determine the likelihood that a certain shopper will be involved in shoplifting. Consistent with the present disclosure, determining the likelihood may include determining a shoplift risk level. The term "shoplift risk level" refers to any indication, numeric or otherwise, of a level (e.g., within a predetermined range) indicative of a probability that a given shopper will attempt to shoplift … the shoplift risk level may have a value between 1 and 10. Alternatively, the shoplift risk level may be expressed as a percentage or any other numerical or non-numerical indication … the system may compare the shoplift risk level to a threshold … the term "threshold" as used herein denotes a reference value, a level, a point, or a range of values. In operation, when a shoplift risk level associated with a shopper exceeds a threshold (or below it, depending on a particular use case), the system may follow a first course of action and, when the shoplift risk level is below it (or above it, depending on a particular use case), the system may follow a second course of action. The value of the threshold may be predetermined for all shoppers or may be dynamically selected based on different considerations … when the shoplift risk level of a certain shopper exceeds a risk threshold, the system may use detail level determination module 2810 and update rate determination module 2812 to reduce the shopping data provided to the certain shopper …”); and
responsive to the number of fraudulent anomalies identified for the user exceeding a threshold, identifying, by the processor of the shopping cart (C/L 8/64-9/16), the user as fraudulent (FIG. 28, item 2820; C/L 121/1-17, “… Action identification module 2820 is configured to use image data from sensors 2801 to determine the likelihood that the shopper will be involved in shoplifting. Specifically, image analysis modules 2820 may detect one or more actions taken by the shopper that may be classified as suspicious and determine a corresponding shoplift risk level … a suspicious action includes any action that may indicate the intent or the act of theft by a shopper. The suspicious action may include any one or more of … a furtive glance by the shopper, a shopper attempt to hide his/her face, a shopper attempt to hide a picked item, and the like … image analysis modules 2820 may detect an avoidance action taken by the shopper to avoid at least one store associate. The detection of the avoidance action may result in a higher determined shoplift risk level than where the shopper is not detected as engaging in such avoidance action …”).
Bronicki discloses a visual indicator of frictionless status of retail shelve. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include a visual indicator of frictionless status, as in Bronicki; to include automated shopping experience using cashier-less systems, as in Khalili; to include monitoring shopping activities using weight data in a store, as in Gu; and to include a shopping cart with weight bump validation, as in Hagen, to improve and/or enhance the technology for an automated shopping cart, as in Sriraman, because it would amount to combining elements that in the combination would perform the same function as they functioned separately. One of ordinary skill in the art before the effective filing date of the invention would have been motivated to combine the references to quickly and accurately resolve an ineligible condition for a product based on a detected ambiguous product interaction event, such as not being able to determine pricing for a product, by utilizing a multitude of sensors, such as a camera, a barcode scanner, and/or a scale (or load sensor) reading the weight of a product.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Cheng (U. S. Patent No. 11328282 B2) – Method And System For Identifying Goods Of Intelligent Shopping Cart
Cheng discloses a method and a system for identifying goods of intelligent shopping cart. The method comprises: reading bar code information of a to-be-purchased goods and obtaining corresponding prestored goods information; continuously detecting and obtaining a total goods weight mn+1 with a total goods weight mn acquired after a previous purchasing action is completed, to obtain a variation mD of the total goods weight. According to the method of the present disclosure, when a customer puts a goods in the shopping cart in the shopping course, the correct goods is automatically identified and recorded in a shopping list, then the customer can directly settle the account after completing the shopping, accordingly a lot of time for the customers to wait for the settlement is saved.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to STEVEN CHISM whose telephone number is (571) 272-5915. The examiner can normally be reached during 9:00 AM – 3:00 PM Monday – Thursday, EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ryan D. Donlon can be reached (571) 270-3602. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/STEVEN CHISM/
Examiner, Art Unit 3692
/RYAN D DONLON/Supervisory Patent Examiner, Art Unit 3692
Prosecution Insights