Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Claims 1-6, 9-13, and 16-20 are presented for examination. Assig: Toshiba. Priority: 10 Jan 2024.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action.
A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-6, 9-13, and 16-20 are rejected under 35 USC 103 as being unpatentable over Sommer et al., U.S. 2017/0243068 in view of Ferreira et al. U.S. 2020/0284883 and Huebler et al., U.S. 5,127,267 and Totoriello et al. U.S. 2018/0364198 and Kanopka et al., U.S. 9,404,899 and Zhu et al., U.S. 2009/0134221.
On claim 1, Sommer cites except as underlined:
A method comprising:
receiving sensor data from one or more acoustic wave sensors, wherein
the one or more acoustic wave sensors transmit acoustic waves towards a set of
items within a receptacle and receive reflected acoustic waves from the set of
items;
figure 3 and [0261] In accordance with various embodiments, recognizing the transport container 400 and/or the spatial location thereof in the image capture region 208 (position determination) can be carried out on the basis of the image data 602. Alternatively or additionally, recognizing the transport container 400 and/or the spatial location thereof in the image capture region 208 can be carried out using an acoustic sensor (e.g. using sound reflection), an optoelectronic sensor (e.g. a light barrier) and/or a radio tag sensor (e.g. RFID—identification with the aid of electromagnetic waves). By way of example, the transport container 400 may include a radio tag (may also be referred to as RFID transponder) and/or optical reflectors (optical markers).
And
[0004] Computer-aided methods of pattern recognition are conventionally used to identify objects arranged in the transport container. In this case, distinctive patterns of the objects are recognized and compared with a database in which the patterns of known objects, e.g. goods, are stored.
extracting an acoustic wave time of flight metric, an amplitude metric,
change in acoustic wave frequency, and acoustic signatures from the received sensor data;
generating, using the extracted acoustic wave time of flight metric,
the amplitude metric, and the change in acoustic wave frequency, a representation for of the set of items within the receptacle based on the sensor data, wherein the representation corresponds to a three-dimensional (3D) model depicting the set of items within the receptable and comprises:
a digital contour of the set of items within the receptable, as indicated
by the amplitude metric and the change in acoustic wave frequency,
wherein the digital contour comprises the physical dimension and shape of
each respective item of the set of items within the receptable, Sommers cites:
a material composition of each respective item of the set of items
within the receptable, as indicated by the acoustic signatures,
and
a spatial arrangement of the set of items within the receptacle as indicated by the acoustic wave time of flight metric:
identifying one or more features for the set of items, comprising:
inputting the representation into an object recognition mode, wherein the object recognition model is trained using a machine learning algorithm to recognize the one or more features of each item in the set of items,
generating, for each item, a matching score indicating a similarity between a respective item and an known item;
retrieving checkout data from one or more checkout devices, the checkout data comprising a transaction list identifying one or more scanned items at the one or more checkout devices;
generating, based on the representation of the set of items, an identification list corresponding to the set of items within the receptable based on the one or more features and matching score associated with each respective item of the set of items;
identifying one or more discrepancies between the set of items
identified from the representation and the one or more scanned items
identified in the transaction list by comparing the identification list with the
transaction list; and
responsive to the identification of the one or more discrepancies, generating an alert
Regarding the excepted: extracting an acoustic wave time of flight metric, an amplitude metric, change in acoustic wave frequency, and acoustic signatures from the received sensor data;
As previously disclosed, Sommer, figure 3, [0004] and [0261] included an embodiment where an acoustic sensor is used to recognize objects in a transport container. However, Sommer doesn’t disclose the excepted claim limitations.
In the related art of pipe location, Huebler, col. 5, lines 37-43 cites
This time delay is the "time of flight" of acoustic signal 6. Depending on the proximity of detectors 4a-4h to pipe 2, the "time of flight" will vary. The shorter the "time of flight," the closer each of detectors 4a-4h is to pipe 2. Detector 4a-4h having the shortest "time of flight" is closest to pipe 2. In this manner, pipe 2 can be precisely located.
Additionally, in the same art of contraband locating, Totoriello [0030] discloses:
Thus, changes in one or more of four easily measurable parameters associated with the passage of a high frequency sound wave through a material transit time, attenuation, scattering, and frequency content can often be correlated with changes in physical properties such as hardness, elastic modulus, density, homogeneity, or grain structure.
(The cited “attenuation” aspect is a quality of an acoustic signal whereupon the acoustic waves, are sent at a certain amplitude and upon its reception by the receiving sensor, the amplitude is diminished responsive to the materials the original wave has encountered)
Furthermore, Totoriello discloses in page 12, claim 1:
A method of non-destructively identifying contraband materials, said method comprising: transmitting an output wave from a crystalline based transducer, said transducer being operatively positioned as an acoustic detector which collects a return of said output acoustic wave; collecting said return; the method further comprising the steps of: counting electrons associated with materials in a target object; analyzing, through a signal processing assembly, frequency changes by comparing said output acoustic wave with said return
Finally, Kanopka discloses:
Col 12, lines 16-28 FIG. 15A shows an exemplary acoustic inspection system 550 including a low frequency source 552 directed at a passenger side of a vehicle 10 and a laser vibrometer 554 directed at a driver side of the vehicle. A handgun 20 is located on the inside of a driver side door of the vehicle. FIG. 15B shows an acoustic image/grid data for an empty car door. FIG. 15C shows a picture of the handgun 20 on the door. FIG. 15D shows an acoustic image of the handgun on the door and registration of the handgun in the picture and acoustic signature. The extent of the handgun 20 is clearly visible in the acoustic image. FIG. 15E is an acoustic image of a gun in a car door after post processing. FIG. 15G shows more detail for a post processed acoustic image of a gun.
It would have been obvious to one of ordinary skill before the effective filing date of the claimed invention to modify Sommer’s embodiment using the features disclosed in Huebler, Totoriello, and Kanopka such that the claimed invention is realize. Huebler discloses using the known detection of an object using “time of flight” principles while Totoriello discloses using attenuation and frequency change of acoustic signals to identify contraband items and Kanopka discloses determining “acoustic signatures” of a detected item. One of ordinary skill, apprised of these known techniques for locating and identifying objects, would have modified Sommer’s embodiment using the features described in Huebler, Totoriello, and Kanopka and the results of the modification would have realized an embodiment meeting the claimed invention.
Regarding the excepted: generating, using the extracted acoustic wave time of flight metric, the amplitude metric, and the change in acoustic wave frequency, a representation for of the set of items within the receptacle based on the sensor data, wherein the representation corresponds to a three-dimensional (3D) model depicting the set of items within the receptable and comprises:
a digital contour of the set of items within the receptable, as indicated
by the amplitude metric and the change in acoustic wave frequency,
wherein the digital contour comprises the physical dimension and shape of
each respective item of the set of items within the receptable, as discussed above, Sommers, as modified by Huebler and Totoriello, disclosed an embodiment employing the principles of “extracted acoustic wave time of flight metric, the amplitude metric, and the change in acoustic wave frequency” to detect, locate, and identify items. However, the modified combination did not disclose a representation or digital contour of the detected items. However, Sommer disclosed:
[0039] In accordance with various embodiments, depth information is furthermore obtained by means of a three-dimensional (3D) image capture. The depth information can be used, for example, to recognize whether an object is situated in different regions of the transport container, e.g. on a lower and/or upper plane. It is thus possible to differentiate in what region of the transport container an object is situated. By way of example, the depth information can be obtained through one of the regions, such that illustratively it is possible to recognize from above whether something is situated in a lower region of the transport container or in a region below the transport container.
Figures 8, 9 and [0185] The checkout system may include a primary screen 802, a barcode scanner 804, a secondary screen 808 and an EC terminal 816. The screen 802 can be configured for outputting the signal. By way of example, it is possible to display on the screen a coloured signal, a geometrical signal and/or an input request, which represents a state of the transport container 400 (here arranged outside the image capture region 208) (e.g. empty or non-empty) and/or a state of the image capture region 208 (e.g. with or without transport container 400).
And
[0209] Alternatively or additionally, in a 3D mode, it is possible to determine whether (and if so how many) pixels of the image data 602 have a depth value (e.g. a distance with respect to the image capture system 202) that deviates from the reference depth information. Optionally, a topography representing the transport container 400 (or, if appropriate, the content thereof) can be determined on the basis of the image data 602.
[0122] The processor can be provided by means of an electronic (programmable) data processing system. The electronic data processing system may furthermore include the data storage medium. By way of example, the electronic data processing system may include or be formed from a microcomputer, e.g. a PC system (Personal Computer System) or a digital checkout system.
Furthermore, Zhu discloses:
[0028] Accordingly, a primary object of the present invention is to provide an improved digital image capturing and processing apparatus for use in POS environments, which are free of the shortcomings and drawbacks of prior art laser scanning and digital imaging systems and methodologies.
It would have been obvious to one of ordinary skill before the effective filing date of the claimed invention to modify the display system disclosed in Sommer using the features previously disclosed in Huebler and Totoriello as well as the digital image capturing features disclosed in Zhu. One of ordinary skill would have wanted Sommer’s visual rendition of the object detection, location, and identification features disclosed in Huebler, Zhu, and Totoriello.
One of ordinary skill would have desired the visual representation for a better point of reference from a user’s perspective.
Regarding the excepted: a material composition of each respective item of the set of items within the receptable, as indicated by the acoustic signatures,
Sommers cites:
[0003] In general, transport containers can be used to transport objects, e.g. goods in the field of production or sales. In this case, it may be necessary to recognize whether something, and if appropriate what, is situated in the transport container, e.g. when registering goods at a checkout exit (this may also be referred to as “Bottom of Basket” recognition—BoB). It is thus possible to reduce costs which arise if unregistered goods pass through the checkout exit without being recognized (this may also be referred to as loss prevention)
And
[0209] Optionally, a topography representing the transport container 400 (or, if appropriate, the content thereof) can be determined on the basis of the image data 602.
And
[0004] Computer-aided methods of pattern recognition are conventionally used to identify objects arranged in the transport container. In this case, distinctive patterns of the objects are recognized and compared with a database in which the patterns of known objects, e.g. goods, are stored.
Sommer doesn’t specifically disclose “material composition” of the respected items. In the same art of object detection, Totoriello cites:
[0030] Accordingly, the motion of any given acoustic wave will be affected by the medium through which it travels. Thus, changes in one or more of four easily measurable parameters associated with the passage of a high frequency sound wave through a material transit time, attenuation, scattering, and frequency content can often be correlated with changes in physical properties such as hardness, elastic modulus, density, homogeneity, or grain structure. As such, acoustic frequency detection according to the present invention may utilize the range of frequencies from approximately 20 KHz to 100 MHz, with most work being performed between 500 KHz and 20 MHz, but in the illustrative case of Nitrogen, will range from 20 MHz-40 MHz, all of which can be easily adjusted depending on the particular element being flagged as further discussed herein. Both longitudinal and shear (transverse) modes of vibration are commonly employed, as well as surface (Rayleigh) waves and plate (Lamb) waves in some specialized cases. Because shorter wavelengths are more responsive to changes in the medium through which they pass, many material analysis applications will benefit from using the highest frequency that the test piece will support.
It would have been obvious to one of ordinary skill before the effective filing date of the claimed invention to modify Sommer’s embodiment using the known material identification features disclosed in Totoriello such that the claimed invention is realized. Totoriello discloses an known embodiment for classifying detected object’s material makeup using the acoustic wave identification feature disclosed in its embodiment. One of ordinary skill would have included this feature into Sommer to provide the user with a more detailed rendering of what is in the cited basket.
Regarding the excepted:
a spatial arrangement of the set of items within the receptacle as indicated by the acoustic wave time of flight metric:
Sommer cites:
[0004] Computer-aided methods of pattern recognition are conventionally used to identify objects arranged in the transport container. In this case, distinctive patterns of the objects are recognized and compared with a database in which the patterns of known objects, e.g. goods, are stored.
And
[0039] In accordance with various embodiments, depth information is furthermore obtained by means of a three-dimensional (3D) image capture. The depth information can be used, for example, to recognize whether an object is situated in different regions of the transport container, e.g. on a lower and/or upper plane. It is thus possible to differentiate in what region of the transport container an object is situated. By way of example, the depth information can be obtained through one of the regions, such that illustratively it is possible to recognize from above whether something is situated in a lower region of the transport container or in a region below the transport container.
Sommer doesn’t disclose the excepted “time of flight” feature.
In the related art of pipe locating, Huebler, col. 5, lines 37-43 discloses:
This time delay is the "time of flight" of acoustic signal 6. Depending on the proximity of detectors 4a-4h to pipe 2, the "time of flight" will vary. The shorter the "time of flight," the closer each of detectors 4a-4h is to pipe 2. Detector 4a-4h having the shortest "time of flight" is closest to pipe 2. In this manner, pipe 2 can be precisely located.
It would have been obvious to one of ordinary skill before the effective filing date of the claimed invention to include into Sommer the “time of flight” measurement feature disclose in Huebler such that the claimed invention is realized. Huebler’s embodiment involves locating pipes using acoustic sensing based on time of flight analysis and one of ordinary skill would have used this known principle of locating an object into Sommer to realize an embodiment meeting the claimed invention.
Regarding the excepted: identifying one or more features for the set of items, comprising: inputting the representation into an object recognition mode, wherein the object recognition model is trained using a machine learning algorithm to recognize the one or more features of each item in the set of items, and generating, for each item, a matching score indicating a similarity between a respective item and an known item;
Sommer doesn’t disclose the excepted claim limitations.
In the similar art of object recognition, Ferreira states:
[4145] The respective data processing characteristics may be assigned to the one or more portions of the sensor data representation 16204 including the one or more objects according to the respective properties of the objects. By way of example, the object recognition process may include a machine learning algorithm and the recognition confidence level may indicate a probability of the correct identification of the object (e.g., the recognition confidence level may be a score value of a neural network). Additionally or alternatively, the object recognition process may be executed in or by a LIDAR system-external device or processor, for example by a sensor fusion box of the vehicle including the LIDAR system 16200.
It would have been obvious to one of ordinary skill before the effective filing date of the claimed invention to include into Sommer the embodiment disclosed in Ferreira such that the claimed invention is realized. Ferreira disclose a known embodiment for not only using a machine learning algorithm to aid in identifying the cited objects while also disclosing a score as to the confidence level of identifying the detected object. One of ordinary skill would have desired this feature to easily identify detected objects while including a scoring system to reassure the user of the truth of the detected item.
Regarding the excepted: generating, based on the representation of the set of items, an identification list corresponding to the set of items within the receptable based on the one or more features and matching score associated with each respective item of the set of items;
Sommer cites:
[0003] In general, transport containers can be used to transport objects, e.g. goods in the field of production or sales. In this case, it may be necessary to recognize whether something, and if appropriate what, is situated in the transport container, e.g. when registering goods at a checkout exit (this may also be referred to as “Bottom of Basket” recognition—BoB). It is thus possible to reduce costs which arise if unregistered goods pass through the checkout exit without being recognized (this may also be referred to as loss prevention).
[0209] Alternatively or additionally, in a 3D mode, it is possible to determine whether (and if so how many) pixels of the image data 602 have a depth value (e.g. a distance with respect to the image capture system 202) that deviates from the reference depth information. Optionally, a topography representing the transport container 400 (or, if appropriate, the content thereof) can be determined on the basis of the image data 602.
Sommer doesn’t specifically disclose the excepted claimed limitations. However, based on the embodiments disclosed in Sommer, it would have been obvious to one of ordinary skill before the effective filing date of the claimed invention to generate the claimed “generating, based on the representation of the set of items, an identification list corresponding to the set of items within the receptable based on the one or more features and matching score associated with each respective item of the set of items.” Sommer already discloses registering goods in the bottom of the basket of a shopping cart. Furthermore, Sommer discloses capturing and displaying on a screen, information regarding contents of a transport container. One of ordinary skill would have provided the claimed “list of items” based on detecting individual items as disclosed in Sommer without the user having to manually inspect the container to list and provide an invoice.
Regarding the excepted:
identifying one or more discrepancies between the set of items
identified from the representation and the one or more scanned items
identified in the transaction list by comparing the identification list with the
transaction list; and responsive to the identification of the one or more discrepancies, generating an alert
Sommer cites:
[0003] In general, transport containers can be used to transport objects, e.g. goods in the field of production or sales. In this case, it may be necessary to recognize whether something, and if appropriate what, is situated in the transport container, e.g. when registering goods at a checkout exit (this may also be referred to as “Bottom of Basket” recognition—BoB). It is thus possible to reduce costs which arise if unregistered goods pass through the checkout exit without being recognized (this may also be referred to as loss prevention)
And
[0209] Optionally, a topography representing the transport container 400 (or, if appropriate, the content thereof) can be determined on the basis of the image data 602.
And
[0004] Computer-aided methods of pattern recognition are conventionally used to identify objects arranged in the transport container. In this case, distinctive patterns of the objects are recognized and compared with a database in which the patterns of known objects, e.g. goods, are stored.
Sommer doesn’t disclose the excepted claim limitations. In the same art of store checkout processes, Zhu discloses:
[0037] Another object of the present invention is to provide such a tunnel-type digital imaging-based system, wherein automatic package identification, profiling/dimensioning, weighing and tracking techniques are employed during self-checkout operations, to reduce checkout inaccuracies and possible theft during checkout operations.
And
[0156] As indicated at Block J, in the event that the total weight of products/goods measured at Block H does correspond with total weight of products measured by the retail tunnel system, then the checkout subsystem 62 automatically generates an alarm or signal advising a retail store supervisor about such weight discrepancies.
It would have been obvious to one of ordinary skill before the effective filing date of the claimed invention to modify Sommer’s embodiment with the inventory checkout feature disclosed in Zhu such that the claimed invention is realized.
Zhu discloses two known embodiments for checking a checkout operation for possible theft: on the first, there is an automatic package identification, profiling/dimensioning, weighing and tracking. On the second, there is an alarm to indicate weight discrepancies. Taken together, one of ordinary skill, apprised of possible indications of theft, would have included an alarm this feature for stop-loss purposes and added this modified feature to Sommer’s checkout system to realize an embodiment meeting the claimed invention.
On claim 2, Sommer and Zhu cites:
The method of claim 1, wherein the one or more features comprise at least one of (i) a number of the set of items within the receptacle; (ii) a shape of an item of the set of items; (iii) a size of an item of the set of items; or (iv) a material composition of an item of the set of items.
[0209] Optionally, a topography representing the transport container 400 (or, if appropriate, the content thereof) can be determined on the basis of the image data 602
Furthermore, Zhu cites:
[0037] Another object of the present invention is to provide such a tunnel-type digital imaging-based system, wherein automatic package identification, profiling/dimensioning, weighing and tracking techniques are employed during self-checkout operations, to reduce checkout inaccuracies and possible theft during checkout operations.
On claim 3, Sommer and Zhu cites:
The method of claim 1, wherein the one or more discrepancies comprises a variance in number, shape, or size of the set of items. See the rejection of claim 1 citing Zhu. The cited “profiling/dimensioning, weighing and tracking techniques” used to deter theft meets the above claim limitations.
[0037] Another object of the present invention is to provide such a tunnel-type digital imaging-based system, wherein automatic package identification, profiling/dimensioning, weighing and tracking techniques are employed during self-checkout operations, to reduce checkout inaccuracies and possible theft during checkout operations.
On claim 4, Sommer cites except as underlined:
The method of claim 1, further comprising extracting depth information for a respective item, of the set of items within the receptacle, based on a respective time of flight, wherein the respective time of flight is measured from a time when the acoustic waves were transmitted towards the respective item to a time when the corresponding reflected acoustic waves were received.
Sommer cites:
figure 3 and [0261] In accordance with various embodiments, recognizing the transport container 400 and/or the spatial location thereof in the image capture region 208 (position determination) can be carried out on the basis of the image data 602. Alternatively or additionally, recognizing the transport container 400 and/or the spatial location thereof in the image capture region 208 can be carried out using an acoustic sensor (e.g. using sound reflection), an optoelectronic sensor (e.g. a light barrier)
Sommer doesn’t disclose the excepted claim limitations. In the related art of pipe location, Huebler, col. 5, lines 37-43 cites
This time delay is the "time of flight" of acoustic signal 6. Depending on the proximity of detectors 4a-4h to pipe 2, the "time of flight" will vary. The shorter the "time of flight," the closer each of detectors 4a-4h is to pipe 2. Detector 4a-4h having the shortest "time of flight" is closest to pipe 2. In this manner, pipe 2 can be precisely located.
It would have been obvious to one of ordinary skill before the effective filing date of the claimed invention to modify Sommer using the time-of-flight location techniques disclosed in Huebler to find an object. One of ordinary skill, apprise of this known feature, would have added this known principle to located objects disclosed in Sommer.
On claim 5, Sommer and Huebler cites:
The method of claim 4, wherein the depth information for the respective item comprises a respective distance between the respective item and the one or more acoustic wave sensors.
[0039] In accordance with various embodiments, depth information is furthermore obtained by means of a three-dimensional (3D) image capture. The depth information can be used, for example, to recognize whether an object is situated in different regions of the transport container, e.g. on a lower and/or upper plane. It is thus possible to differentiate in what region of the transport container an object is situated. By way of example, the depth information can be obtained through one of the regions, such that illustratively it is possible to recognize from above whether something is situated in a lower region of the transport container or in a region below the transport container.
Furthermore, as shown in the rejection of claim 4, Huebler, col. 5, lines 37-43 cites
This time delay is the "time of flight" of acoustic signal 6. Depending on the proximity of detectors 4a-4h to pipe 2, the "time of flight" will vary. The shorter the "time of flight," the closer each of detectors 4a-4h is to pipe 2. Detector 4a-4h having the shortest "time of flight" is closest to pipe 2. In this manner, pipe 2 can be precisely located.
On claim 6, Sommer cites:
The method of claim 1, wherein the model comprises a three-dimensional representation of the set of items within the receptacle.
[0039] In accordance with various embodiments, depth information is furthermore obtained by means of a three-dimensional (3D) image capture. The depth information can be used, for example, to recognize whether an object is situated in different regions of the transport container, e.g. on a lower and/or upper plane.
Claim 9 is rejected for the same reasons disclosed in the rejection of claim 1 over Sommer in view of Ferreira and Huebler and Totoriello and Kanopka and Zhu.
On claim 10, Sommer and Huebler cites:
The system of claim 9, wherein the one or more features comprise at least one of (i) a number of the set of items within the receptacle; (ii) a shape of an item of the set of items; (iii) a size of an item of the set of items; or (iv) a material composition of an item of the set of items.
[0209] Optionally, a topography representing the transport container 400 (or, if appropriate, the content thereof) can be determined on the basis of the image data 602
Merriam Webster’s Dictionary defines “topography” as
“2 a: the configuration of a surface including its relief and the position of its natural and man-made features.”
“Topography” conforms to item (ii) regarding the shape of an item of the set of items.
Furthermore, regarding the art of pipe location, Huebler, col. 5, lines 37-43 cites
This time delay is the "time of flight" of acoustic signal 6. Depending on the proximity of detectors 4a-4h to pipe 2, the "time of flight" will vary. The shorter the "time of flight," the closer each of detectors 4a-4h is to pipe 2. Detector 4a-4h having the shortest "time of flight" is closest to pipe 2. In this manner, pipe 2 can be precisely located.
See the rejection of claim 9 which discloses these limitations.
On claim 11, Sommer and Huebler cites:
The system of claim 9, wherein the program, which, when executed on any combination of the one or more processors, performs the operations further comprising extracting depth information for a respective item, of the set of items within the receptacle, based on a respective time of flight, wherein the respective time of flight is measured from a time when the acoustic waves were transmitted towards the respective item to a time when the corresponding reflected acoustic waves were received. See the rejection of claim 9 citing Huebler and the “time of flight” feature used to locate pipes.
On claim 12, Sommer and Huebler cites:
The system of claim 11, wherein the depth information for the respective item comprises a respective distance between the respective item and the one or more acoustic wave sensors. See the rejection of claim 4 citing Huebler, col. 5, lines 37-43 cites using “time of flight” acoustic signaling to locate a pipe.
On claim 13, Sommer cites: The system of claim 9, wherein the representation comprises a three-dimensional model of the set of items within the receptacle.
[0039] In accordance with various embodiments, depth information is furthermore obtained by means of a three-dimensional (3D) image capture. The depth information can be used, for example, to recognize whether an object is situated in different regions of the transport container, e.g. on a lower and/or upper plane. It is thus possible to differentiate in what region of the transport container an object is situated. By way of example, the depth information can be obtained through one of the regions, such that illustratively it is possible to recognize from above whether something is situated in a lower region of the transport container or in a region below the transport container.
On claim 16, Sommer cites:
One or more non-transitory computer-readable media containing, in any combination, computer program code that, when executed by operation of a computer system,
[0128] The device 200 may include an optical image capture system 202, a data storage medium 204 and a processor 206. The processor 206 can be coupled to the optical image capture system 202 and the data storage medium 204, e.g. by means of a data line (i.e. such that data can be transferred between them)
And
[0283] The initial phase 2101 may optionally include in 2201 (may also be referred to as program start 2101): starting a program that is configured to carry out a method in accordance with various embodiments. By way of example, a processor can be put into a state ready for operation. The processor can be configured to carry out the method in accordance with various embodiments, e.g. by virtue of said processor executing the program.
Furthermore, claim 16 is rejected for the same reasons disclosed in the rejection of claim 1 over Sommer in view of Ferreira and Huebler and Totoriello and Kanopka and Zhu.
On claim 17, Sommer and Zhu cites:
The one or more non-transitory computer-readable media of claim 16, wherein the one or more features comprise at least one of (i) a number of the set of items within the receptacle; (ii) a shape of an item of the set of items; (iii) a size of an item of the set of items; or (iv) a material composition of an item of the set of items.
[0209] Optionally, a topography representing the transport container 400 (or, if appropriate, the content thereof) can be determined on the basis of the image data 602
Merriam Webster’s Dictionary defines “topography” as
“a: the configuration of a surface including its relief and the position of its natural and man-made features.”
“Topography” conforms to item (ii) regarding the shape of an item of the set of items.
Furthermore, Zhu discloses:
[0037] Another object of the present invention is to provide such a tunnel-type digital imaging-based system, wherein automatic package identification, profiling/dimensioning, weighing and tracking techniques are employed during self-checkout operations, to reduce checkout inaccuracies and possible theft during checkout operations.
See the rejection of claim 16 which discloses these limitations.
On claim 18, Sommer and Huebler cites:
The one or more non-transitory computer-readable media of claim 16, wherein the computer program code that, when executed by operation of a computer system, performs the operations further comprising extracting depth information for a respective item, of the set of items within the receptacle, based on a respective time of flight, wherein the respective time of flight is measured from a time when the acoustic waves were transmitted towards the respective item to a time when the corresponding reflected acoustic waves were received.
See the rejection of claim 16 citing Huebler, col. 5, lines 37-43:
Furthermore, regarding the art of pipe location, Huebler, col. 5, lines 37-43 cites
This time delay is the "time of flight" of acoustic signal 6. Depending on the proximity of detectors 4a-4h to pipe 2, the "time of flight" will vary. The shorter the "time of flight," the closer each of detectors 4a-4h is to pipe 2. Detector 4a-4h having the shortest "time of flight" is closest to pipe 2. In this manner, pipe 2 can be precisely located.
On claim 19, Sommer cites:
The one or more non-transitory computer-readable media of claim 18, wherein the depth information for the respective item comprises a respective distance between the respective item and the one or more acoustic wave sensors.
[0039] In accordance with various embodiments, depth information is furthermore obtained by means of a three-dimensional (3D) image capture. The depth information can be used, for example, to recognize whether an object is situated in different regions of the transport container, e.g. on a lower and/or upper plane. It is thus possible to differentiate in what region of the transport container an object is situated. By way of example, the depth information can be obtained through one of the regions, such that illustratively it is possible to recognize from above whether something is situated in a lower region of the transport container or in a region below the transport container.
On claim 20, Sommer cites:
The one or more non-transitory computer-readable media of claim 16, wherein the representation comprises a three-dimensional model of the set of items within the receptacle.
[0209] Alternatively or additionally, in a 3D mode, it is possible to determine whether (and if so how many) pixels of the image data 602 have a depth value (e.g. a distance with respect to the image capture system 202) that deviates from the reference depth information. Optionally, a topography representing the transport container 400 (or, if appropriate, the content thereof) can be determined on the basis of the image data 602.
Response to Arguments
The applicant’s argument regarding claim 1’s amended limitations include:
“…the references appear to be entirely silent with respect to generating a “representation” based on acoustic information, where the representation corresponds to “three-dimensional (3D) modeling.”
Claim 1’s limitations include, in part, “generating, using the extracted acoustic wave time of flight metric, the amplitude metric, and the change in acoustic wave frequency, a representation for of the set of items within the receptacle based on the sensor data, wherein the representation corresponds to a three-dimensional (3D) model depicting the set of items within the receptable…”
However, limitations regarding “3D modeling” or any issues related to three-dimensional renderings were not discussed in the previous Office Action, the amendment now requiring a new search and consideration, making the applicant’s arguments moot. Furthermore, as indicated in Sommer discloses a “3D mode.”
The applicant’s arguments regarding claim 1 further include:
“Additionally, in rejecting the pre-amended claims, the Office argues that Sommer teaches "generating a representation of the set of items" based on sensor data because Sommer discusses "recognizing [a] transport container [] and/or the spatial location thereof," and notes that this recognition may be performed "using an acoustic sensor (e.g., using sound reflection)." Non-Final Office Action, p. 3. However, Applicant notes that there is a clear difference between recognizing the position of a "transport container," as discussed in Sommer, and generating a representation corresponding to "three-dimensional (3D) model depicting the set of items within the receptacle," as well as indicating the "spatial arrangement" of items in such a container, as recited in the present claims.”
The examiner has carefully reviewed this argument and disagrees. As previously cited, Sommer disclosed:
[0209] Alternatively or additionally, in a 3D mode, it is possible to determine whether (and if so how many) pixels of the image data 602 have a depth value (e.g. a distance with respect to the image capture system 202) that deviates from the reference depth information. Optionally, a topography representing the transport container 400 (or, if appropriate, the content thereof) can be determined on the basis of the image data 602.
Furthermore, Sommer discloses:
[0007] Furthermore, a conventional recognition of the content of a transport container may be limited and/or inaccurate, e.g. in the case of flat objects (which have a small cross section e.g. in one orientation); objects having little or no accentuation and/or colouration, e.g. having only low information, colour and/or texture content (for example a virtually homogenous layout such as black, white, grey, etc.); and/or transparent objects. Furthermore, objects not stored in the database are regularly not recognized or recognized incorrectly.
These are clear references to items located within the cited container since the word “content” means whatever items happens to be inside the container.
The applicant’s argument regarding the rejection of claim 1 also included:
“Further, the Examiner notes that Sommer mentions that a "topography representing the transport container 400 (or, if appropriate, the content thereof) can be determined on the basis of image data 602." Id. at 4. That is, Sommer briefly contemplates determining a "topography" of a transport container. As Sommer explains, this "topography" represents a "three-dimensional profile (e.g. an area in the position space)," such that "it is possible to determine for example a spatial location (e.g. position, alignment and/or distance with respect to a reference point) in the position space." [0114]. However, Applicant notes that Sommer explicitly contemplates determining this "topography" using image data, and does not appear to teach or suggest determining a topography using acoustic data. Moreover, Applicant submits that a "topography," as contemplated by Sommer, does not appear to teach or suggest a representation indicating the "contour" of a set of items, or the "spatial arrangement" of the set of items, within a receptacle, as recited by the present claims.”
This argument refers to the claimed “generating, using the extracted acoustic wave time of flight metric, the amplitude metric, and the change in acoustic wave frequency, a representation for of the set of items within the receptacle based on the sensor data, wherein the representation corresponds to a three-dimensional (3D) model depicting the set of items within the receptable and comprises: a digital contour of the set of items within the receptable, as indicated by the amplitude metric and the change in acoustic wave frequency, wherein the digital contour comprises the physical dimension and shape of each respective item of the set of items within the receptable.”
However, as was previously discussed under item 6, the claimed “3D” model was not previously examined. Furthermore, the rejection of claim 1 further includes the 3D rendering as it relates to acoustic properties. For this reason, the applicant’s arguments are moot.
Claim 1 includes the limitations: “a digital contour of the set of items within the receptable, as indicated by the amplitude metric and the change in acoustic wave frequency, wherein the digital contour comprises the physical dimension and shape of
each respective item of the set of items within the receptable.”
The applicant’s argument asserts:
“Moreover, as discussed above, the claims have been amended to recite, in part, generating a representation that includes "a digital contour of the set of items within the receptacle, as indicated by the amplitude metric and the change in acoustic wave frequency, and a spatial arrangement of the set of items within the receptacle as indicated by the acoustic wave time of flight metric." Applicant respectfully submits that the cited references fail to teach or suggest generating a representation of the "digital contour of the set of items as indicated by the amplitude metric and the change in acoustic wave frequency." The references are similarly silent with respect to determining the "spatial arrangement of the set of items as indicated by the acoustic wave time of flight metric."
The examiner has carefully reviewed the applicant’s arguments. The claimed “digital contour of the set of items as indicated by the amplitude metric and the change in acoustic wave frequency” was not previously examined, that is, the claimed “digital contour” as processed by using the amplitude metric and change in acoustic frequency” as not previously discussed. Accordingly, the applicant’s argument regarding these limitations are moot since the amendments requires a new search and consideration, as provided for in the rejection of claim 1 under Sommers and Zhu.
The applicant’s arguments further include:
‘Additionally, claims 1, 9, and 16 have been amended to recite, in part, "retrieving checkout data from one or more checkout devices, the checkout data comprising a transaction list identifying one or more scanned items at the one or more checkout devices," "generating, based on the representation of the set of items, an identification list corresponding to the set of items within the receptacle based on the one or more features and the matching score associated with each respective item of the set of items," and "identifying one or more discrepancies between the set of items identified from the representation and the one or more scanned items identified in the transaction list by comparing the identification list with the transaction list." Applicant respectfully submits that the cited references fail to teach or suggest at least these elements of the present claims.”
The examiner has carefully reviewed the applicant’s arguments. The claimed
"generating, based on the representation of the set of items, an identification list corresponding to the set of items within the receptacle based on the one or more features and the matching score associated with each respective item of the set of items," and "identifying one or more discrepancies between the set of items identified from the representation and the one or more scanned items identified in the transaction list by comparing the identification list with the transaction list" was not previously examined.
Furthermore and relatedly, the applicant argues the claimed “checkout system” is not analogous to the cited “scanner” included in the rejection of pre-amended claim 7. However, the examiner also cites Sommer:
[0122] The processor can be provided by means of an electronic (programmable) data processing system. The electronic data processing system may furthermore include the data storage medium. By way of example, the electronic data processing system may include or be formed from a microcomputer, e.g. a PC system (Personal Computer System) or a digital checkout system.
The implication of the cited “digital checkout system” includes a list of items scanned. This does not include the modification of Sommer in view of Zhu, which is part of the rejection of claim 1.
Accordingly, the applicant’s argument regarding these limitations are moot since the amendments requires a new search and consideration, as provided for in the rejection of claim 1 under Sommers and Zhu.
The applicant’s argument further includes:
Similarly, in rejecting pre-amended claim 8, the Office argues that Sommer
teaches "generating an identification list of items from the representation by comparing the identified one or more features to acoustic signatures that correspond to known items," again pointing to Sommer's brief mention of a "scanner" and "printing out an invoice." Id. at 15. Specifically, the Office argues that "[t]he checkout keyboard is asserted to input items at checkout which is recorded on the invoice." Id. However, Applicant respectfully submits that there is a clear difference between scanning or entering items, as the Office suggests, and generating an identification list "based on the representation of the set of items," much less doing so "based on the one or more features and the matching score associated with each respective item of the set of items," as recited by the amended claims.
However, due to the known embodiments disclosed in Sommer, and as pointed out in the rejection of these particular limitations, one of ordinary skill would have realized the claimed list based on the representative of set of items as seen in the rejection. Accordingly, the applicant’s arguments regarding these limitations are unpersuasive.
Regarding the excepted: “identifying one or more discrepancies between the set of items identified from the representation and the one or more scanned items identified in the transaction list by comparing the identification list with the transaction list; and
responsive to the identification of the one or more discrepancies, generating an alert,” the applicant alleges:
“Furthermore, the Office concedes that Sommer fails to teach "comparing the identification list with the transaction list to detect the discrepancy," but asserts that Schneider remedies this deficiency. Id. at 15-16. However, as the Office explains, Schneider describes a system where a "weight disparity between the customer's weight entering the store and later" in order to identify suspected shoplifting. Id. Respectfully, Applicant submits that evaluating a weight discrepancy clearly does not teach or suggest "identifying one or more discrepancies between the set of items identified from the representation and the one or more scanned items identified in the transaction list," as recited in the amended claims.”
The examiner has carefully reviewed the applicant’s arguments. While the rejection of claim 1 under these limitations has been further amended to better meet the claim limitations, Schneider remains good art as it applies to this particular set of claim limitations. Schneider operates from an expect set of weights on listed items and if there are any weight disparities, one of ordinary skill would assume the differences (likely the actual weight being more than the expect weight). However, since the scope of the entirety of claim 1 was changed as previously discussed, this aspect of claim 1’s rejection has been amended using Sommer in view of Zhu to better obviate the claim limitations.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CAL EUSTAQUIO whose telephone number is (571)270-7229. The examiner can normally be reached on 8am-5pm. If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Brian Zimmerman, can be reached at (571) 272-3059. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of an application may be obtained from the Patent Application lnformation Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAlR only. For more information about the PAlR system, see http:/lpair-direct.uspto.gov. Should you have questions on access to the Private PAlR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CAL J EUSTAQUIO/Examiner, Art Unit 2686
/BRIAN A ZIMMERMAN/Supervisory Patent Examiner, Art Unit 2686
Prosecution Insights