DETAILED ACTION
This is a Non-final office action on the merits. The U.S. Patent and Trademark Office (the Office) has received claims 1-19 in application number 18/630,663. Claims 1-19 are pending and have been examined on the merits.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 2-19 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 2, 3, 7, 10, 13, 14, 16, 18 and 19 recite the limitation "the first shelf". Claims 2 and 13 are dependent on Independent Claim 1 but Claim 1 does not recite “a first shelf” thus there is insufficient antecedent basis for the limitation "the first shelf" in these claims. Claims 18 and 19 are independent and recite "the first shelf" with no prior reference to “a first shelf” thus there is insufficient antecedent basis for the limitation "the first shelf" in these claims. Claims 3, 7, 10, 14 and 16 recite “the first shelf” with no “a first shelf” in their chain of dependency thus there is insufficient antecedent basis for the limitation "the first shelf" in these claims. Further, Claims 3-10 and 14-17 are rejected for being dependent on a rejected claim.
Claims 2, 3, 6, 9, 11, 12, 13, 14, 16, 17, 18 and 19 recite the limitation "the second shelf". Claims 2, 11, 12 and 13 are dependent on Independent Claim 1 but Claim 1 does not recite “a second shelf” thus there is insufficient antecedent basis for the limitation "the second shelf" in these claims. Examiner notes that Claim 13 recites “a second shelf” but recites it *after* the first reference to “the second shelf” making the first reference to “the second shelf” improper but the subsequent references to “the second shelf” correct. Claims 18 and 19 are independent and recite "the second shelf" with no prior reference to “a second shelf” thus there is insufficient antecedent basis for the limitation "the second shelf" in the claims. Claims 3, 6, 9, 14, 16 and 17 recite “the first shelf” with no “a first shelf” in their chain of dependency thus there is insufficient antecedent basis for the limitation "the first shelf" in these claims. Further, Claims 3-10 and 14-17 are rejected for being dependent on a rejected claim.
Appropriate correction is required.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-5, 7-10 and 18-19 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by U.S. Patent Publication 2022/0303445 (Williams).
Regarding Claim 1:
Williams teaches a robot that moves around a store creating a real-o-gram using cameras. Williams teaches: A shelf monitoring moving robot moving within a store in which a plurality of shelves are disposed and monitoring products and electronic shelf labels arranged on the plurality of shelves, the shelf monitoring moving robot comprising: a body… an imaging device provided on the body to take images of the plurality of shelves disposed within the store; ([0013] “an autonomous robot to move along an aisle that is lined with shelves or other fixtures capable of holding inventory or products, with the autonomous robot acting as a movable base for multiple cameras”).
a driving module providing power for movement of the body; [0083] “drive base 540” and [0123] “drivetrain components of the movable platform or robot”).
an electronic shelf label identification device provided on the body to obtain identification information of electronic shelf labels provided on the plurality of shelves; ([0139] “the robot or movable platform could come into proximity with an electronic device, such as an electronic shelf label (ESL) or smart shelf. The robot or movable platform can charge these electronic devices inductively or via RFID, collect data wirelessly from these electronic devices” and [0073] “The realogram uses shelf labels, barcodes, and product identification databases to identify products, localize product placement, estimate product count, count the number of product facings, or even identify or locate missing products”).
and a controller controlling the operation of the driving module, the imaging device, and the electronic shelf label identification device and generating realogram data including display status of products provided on the plurality of shelves based on images of the plurality of shelves captured by the imaging device. ([0061-0062] “An electronic control unit 120 contains an autonomous robot sensing and navigation control module 124 that manages robot responses. Robot position localization may utilize external markers and fiducials, or… simultaneous localization and mapping… The electronic control unit 120 also provides image processing using a camera control and data processing module 122. Autonomous robot sensing and navigation control module 124 manages robot responses, and communication module 126 manages data input and output… The communication module 126 is connected to the processing module 125 to transfer realogram data to remote locations, including store servers or other supported camera systems, and additionally receive inventory information to aid in realogram construction”).
Regarding Claim 2:
Williams teaches all of the elements of Claim 1. Williams also teaches: The robot of claim 1, wherein the controller is configured to: control the driving module to move the body between the first shelf and the second shelf facing each other among the plurality of shelves to a first position closer to the second shelf than to the first shelf, control the imaging device to capture a first image of the first shelf at the first position ([0063] “each robot follows path 205 along the length of an aisle, with multiple cameras capturing images of the shelves 202…Using absolute location sensors, relative distance measurements to the shelves, triangulation to a known landmark, conventional simultaneous localization and mapping (SLAM) methodologies, or relying on beacons positioned at known locations in a blueprint or a previously built map, the robots 230 and 232 can move along a path generally parallel to shelves 202” and see [figure 2] showing robots closer to one shelf than the other.)
and obtain information about the first position. ([0058] “The object sensing suite includes forward 133, side 134 and 135, top 132 and rear (not shown) image and depth sensors to aid in object detection, localization, and navigation… sensors capable of detecting electromagnetic, light, or other location beacons can be useful for precise positioning of the autonomous robot” and see [0061]).
Regarding Claim 3:
Williams teaches all of the elements of Claims 1 and 2. Williams also teaches: The robot of claim 2, wherein the controller is further configured to: control the driving module to move the body from the first position to a second position separated by a predetermined distance in a direction parallel to the longitudinal direction of the second shelf, control the imaging device to obtain a second image of the first shelf at the second position ([0063] “each robot follows path 205 along the length of an aisle, with multiple cameras capturing images of the shelves 202…Using absolute location sensors, relative distance measurements to the shelves, triangulation to a known landmark, conventional simultaneous localization and mapping (SLAM) methodologies, or relying on beacons positioned at known locations in a blueprint or a previously built map, the robots 230 and 232 can move along a path generally parallel to shelves 202” and see [figure 2] showing a robot closer to one shelf than the other.)
and obtain information about the second position. ([0058] “The object sensing suite includes forward 133, side 134 and 135, top 132 and rear (not shown) image and depth sensors to aid in object detection, localization, and navigation… sensors capable of detecting electromagnetic, light, or other location beacons can be useful for precise positioning of the autonomous robot” and see [0061]).
Regarding Claim 4:
Williams teaches all of the elements of Claims 1-3. Williams also teaches: The robot of claim 3, wherein the controller is further configured to match the first image to information about the first position and match the second image to information about the second position. ([0063] “The depth map is registered onto the images captured by the shelf cameras, so the location of each pixel on target can be estimated in 3D. Using available information, consecutive images can be stitched together at the correct depth to create accurate panoramic images that spans an entire shelving unit” and [0093] “pre-processed data along with coordinates and images from a single depth (e.g., front of the shelf) to a main processing unit that will combine images from all cameras into a single panorama (at a single depth, e.g., front of the shelf) and cross-correlate the locations of the scanned barcodes to objects in the panorama”).
Regarding Claim 5:
Williams teaches all of the elements of Claims 1-3. Williams also teaches: The robot of claim 3, wherein, in obtaining information about the first position and information about the second position, the controller is configured to obtain the information about the first position and the information about the second position by performing wireless communication with an external electronic device. ([0058] “sensors capable of detecting electromagnetic, light, or other location beacons can be useful for precise positioning of the autonomous robot” and see [0061]).
Regarding Claim 7:
Williams teaches all of the elements of Claims 1-4. Williams also teaches: The robot of claim 4, wherein the controller is further configured to stitch the first image and the second image adjacent to each other to generate realogram data including display status of products displayed on the first shelf based on data generated by matching the first image with the information about the first position and matching the second image with the information about the second position. ([0063] “The depth map is registered onto the images captured by the shelf cameras, so the location of each pixel on target can be estimated in 3D. Using available information, consecutive images can be stitched together at the correct depth to create accurate panoramic images that spans an entire shelving unit” and [0093] “pre-processed data along with coordinates and images from a single depth (e.g., front of the shelf) to a main processing unit that will combine images from all cameras into a single panorama (at a single depth, e.g., front of the shelf) and cross-correlate the locations of the scanned barcodes to objects in the panorama”).
Regarding Claim 8:
Williams teaches all of the elements of Claims 1-4 and 7. Williams also teaches: The robot of claim 7, wherein, when stitching the first image with the second image, the controller is configured to extract an overlapping area between the first image and the second image using a feature extraction algorithm and stitch the first image with the second image so that the overlapping area is not repeatedly accounted for. ([0130] “the unused images or portions of images can be discarded to prevent further resources (e.g., memory, storage, CPU cycles, power) from being dedicated to the unused images or images of less interest. For example, if images of the same or similar frame are captured at three different focus depths, one-third of those images can be discarded after the image or portions of the image of interest has been identified and saved. In some embodiments, a panoramic stitching process can be performed after images of less interest are discarded”).
Regarding Claim 9:
Williams teaches all of the elements of Claims 1 and 2. Williams also teaches: The robot of claim 2, further including a distance sensor provided on the body to measure the distance from the body to the second shelf, wherein, in controlling the driving module to move the body to the first position, the controller is configured to control the driving module to move the body to a position within a predetermined threshold distance from the second shelf based on the distance information from the body to the second shelf measured by the distance sensor. ([0063] “one range finding sensor to measure distance between the multiple cameras and the shelves and products on shelves, with an accuracy of less than 5 centimeters, and with a typical accuracy range between about 5 centimeters and 1 millimeters. As will be appreciated, LIDAR or other range sensing instruments with similar accuracy can also be used in selected applications. Using absolute location sensors, relative distance measurements to the shelves, triangulation to a known landmark, conventional simultaneous localization and mapping (SLAM) methodologies, or relying on beacons positioned at known locations in a blueprint or a previously built map, the robots 230 and 232 can move along a path generally parallel to shelves 202… consecutive images can be stitched together at the correct depth to create accurate panoramic images that spans an entire shelving unit”).
Regarding Claim 10:
Williams teaches all of the elements of Claims 1-3. Williams also teaches: The robot of claim 3, further including a distance sensor provided on the body to measure the distance from the body to the first shelf, wherein, in controlling the driving module to move the body to the second position from the first position, the controller is configured to determine the predetermined distance between the first position and the second position based on field-of-view information of the imaging device and the distance information from the body to the first shelf measured by the distance sensor. ([0059] “one or more shelf units or fixtures (e.g., target 102) would be imaged by a diverse set of camera types, including downwardly (142 and 144) or upwardly (143 and 148) fixed focal length cameras that cover a defined field of view less than the whole of a target shelf unit; a wide field of view camera 145 to provide greater photographic coverage than the fixed focal length cameras; and a variable field of view, zoomable and focusable camera 146 to capture barcodes, product identification numbers, and shelf labels. Alternatively, a high resolution, tilt controllable camera can be used to identify shelf labels. These camera 140 derived images can be stitched together, with products in the images identified, and position determined” and see [0063]).
Regarding Claim 11:
Williams teaches all of the elements of Claim 1. As best understood in light of the 35 USC 112b rejection, above, Williams also teaches: The robot of claim 1, wherein the electronic shelf label identification device includes a wireless communication module that obtains data of identification information of the electronic shelf label through wireless communication with the electronic shelf label provided on the second shelf. ([0139] “the robot or movable platform could come into proximity with an electronic device, such as an electronic shelf label (ESL) or smart shelf. The robot or movable platform can charge these electronic devices inductively or via RFID, collect data wirelessly from these electronic devices” and [0073] “The realogram uses shelf labels, barcodes, and product identification databases to identify products, localize product placement, estimate product count”). Examiner is interpreting that “the second shelf”, as currently claimed, can be any shelf.
Regarding Claim 12:
Williams teaches all of the elements of Claim 1. As best understood in light of the 35 USC 112b rejection, above, Williams also teaches: The robot of claim 1, wherein the electronic shelf label identification device includes an additional imaging device that takes an image of one of the electronic shelf labels provided on the second shelf; ([0059] “diverse set of camera types, including downwardly (142 and 144) or upwardly (143 and 148) fixed focal length cameras that cover a defined field of view less than the whole of a target shelf unit; a wide field of view camera 145 to provide greater photographic coverage than the fixed focal length cameras; and a variable field of view, zoomable and focusable camera 146 to capture barcodes, product identification numbers, and shelf labels”).
and the controller is further configured to identify a pattern code displayed on one of the electronic shelf labels from an image captured by the additional imaging device and obtain data of the identification information of one of the electronic shelf label assigned to the identified pattern code. (see at least [0073] “barcodes”). Examiner is interpreting that “the second shelf”, as currently claimed, can be any shelf.
Regarding Claim 13:
Williams teaches all of the elements of Claim 1. As best understood in light of the 35 USC 112b rejection, above, Williams also teaches: The robot of claim 1, wherein the controller is further configured to: control the driving module to move the body between the first shelf and the second shelf facing each other among the plurality of shelves to a first position closer to a second shelf than to the first shelf… control the driving module to move the body between the first shelf and the second shelf to a second position closer to the first shelf than to the second shelf, (see [0063] and [Fig 2]).
control the imaging device to capture a first image of the first shelf at the first position… control the imaging device to capture a second image of the second shelf at the second position, (see at least [0063]).
control the electronic shelf identification device to obtain, at the first position, identification information of a first electronic shelf label adjacent to the first position among a plurality of electronic shelf labels provided on the second shelf… and control the electronic shelf identification device to obtain, at the second position, identification information of a second electronic shelf label adjacent to the second position among a plurality of electronic shelf labels provided on the first shelf. [0073] “The realogram uses shelf labels, barcodes, and product identification databases to identify products, localize product placement, estimate product count”). Examiner is interpreting that “the second shelf”, as currently claimed, can be any shelf.
Regarding Claim 14:
Williams teaches all of the elements of Claims 1 and 13. As best understood in light of the 35 USC 112b rejection, above, Williams also teaches: The robot of claim 13, wherein the controller is further configured to: generate first realogram data including display status of products provided on the first shelf based on the first image, and generate second realogram data including display status of products provided on the second shelf based on the second image. Examiner is interpreting Applicant’s “display status of products “to mean “the current product quantities” and this interpretation is reasonable in view of Applicant’s specification at [0007]. Williams teaches this: ([0009] “In addition to realogram mapping, this system can be used detect out of stock products, estimate depleted products, estimate amount of products …estimate or count the number of product facings”). Examiner is interpreting that “the second shelf”, as currently claimed, can be any shelf.
Regarding Claim 15:
Williams teaches all of the elements of Claims 1, 13, and 14. As best understood in light of the 35 USC 112b rejection, above, Williams also teaches: The robot of claim 14, wherein, in generating the first realogram data, the controller is configured to include product-related information assigned to the second electronic shelf label corresponding to the identification information of the second electronic shelf label obtained by the electronic shelf label identification device in the first realogram data, and in generating the second realogram data, the controller is configured to include product-related information assigned to the first electronic shelf label corresponding to the identification information of the first electronic shelf label obtained by the electronic shelf label identification device in the second realogram data. Williams also teaches this: ([0009] “In addition to realogram mapping, this system can be used detect out of stock products, estimate depleted products, estimate amount of products …estimate or count the number of product facings”). Examiner is interpreting that “the second shelf”, as currently claimed, can be any shelf.
Regarding Claim 18:
Williams teaches: A shelf monitoring system comprising: a shelf monitoring moving robot moving within a store in which a plurality of shelves are disposed and monitoring products and electronic shelf labels provided on the plurality of shelves; ([0013] “an autonomous robot to move along an aisle that is lined with shelves or other fixtures capable of holding inventory or products, with the autonomous robot acting as a movable base for multiple cameras”).
and a server including at least one processor performing computations for controlling the operation of the shelf monitoring moving robot and a memory storing commands and programs for controlling the operation of the shelf monitoring moving robot, ([0010] “the movable base can be a tele-operated robot or an autonomous robot capable of guiding itself through a store or warehouse” and [0062] “store servers”).
wherein the at least one processor is configured to: control the shelf monitoring moving robot between the first shelf and the second shelf facing each other among the plurality of shelves to sequentially move through a plurality of positions closer to the second shelf than to the first shelf, ([0063] and see [figure 2] showing robots closer to one shelf than the other.)
control an imaging device of the shelf monitoring moving robot to obtain a plurality of images by taking images of the first shelf sequentially at each of the plurality of positions, obtain information about the plurality of positions, ([0063] and [0058]).
match an image of the first shelf obtained at any one position of the plurality of positions to information about the any one position and stores the matched information in the memory. ([0063] and [0093]).
Regarding Claim 19:
Williams teaches: A shelf monitoring method for a shelf monitoring moving robot to move within a store in which a plurality of shelves are disposed and monitor products and electronic shelf labels provided on the plurality of shelves, ([0013] “an autonomous robot to move along an aisle that is lined with shelves or other fixtures capable of holding inventory or products, with the autonomous robot acting as a movable base for multiple cameras”).
the method comprising: moving sequentially between the first and second shelves included in the plurality of shelves and facing each other through a plurality of positions closer to the second shelf than to the first shelf by the shelf monitoring moving robot; ([0063] and see [figure 2] showing robots closer to one shelf than the other.)
taking images of the first shelf sequentially at each of the plurality of positions by the shelf monitoring moving robot; obtaining information about the plurality of positions, ([0063] and [0058]).
and matching an image of the first shelf obtained at any one position among the plurality of positions to information about the any one position. ([0063] and [0093]).
Examiner’s Comment
Regarding Claim 11:
The robot of claim 1, wherein the electronic shelf label identification device includes a wireless communication module that obtains data of identification information of the electronic shelf label through wireless communication with the electronic shelf label provided on the second shelf.
As currently claimed, Claim 11 is dependent on claim 1 and not on claim 2. As previously discussed, Claim 2 is rejected for being indefinite because it lacks proper antecedent basis. Claim 11 recites “the second shelf” and from the context of Applicant’s specification, Examiner believes that Applicant means that “the second shelf” is the shelf behind the robot and not the shelf that it is facing and photographing. Claim 2 limits Claim 1 to shelves facing each other and shelves claimed in Claim 11, without this limitation, could mean any shelves. Williams does not teach that a robot is in communication with the electronic shelf label “on the second shelf” *with the additional limitation that the second shelf label is behind the robot*. After a diligent search Examiner is not able to find art that teaches this particular combination of elements. Examiner holds that, depending on specific amended claim construction, a claim that is a combination of Claims 1, 2 and 11 could be allowable over prior art.
Regarding Claim 12:
The robot of claim 1, wherein the electronic shelf label identification device includes an additional imaging device that takes an image of one of the electronic shelf labels provided on the second shelf; and the controller is further configured to identify a pattern code displayed on one of the electronic shelf labels from an image captured by the additional imaging device and obtain data of the identification information of one of the electronic shelf label assigned to the identified pattern code.
As currently claimed, Claim 12 is dependent on claim 1 and not on claim 2. As previously discussed, Claim 2 is rejected for being indefinite because it lacks proper antecedent basis. Claim 12 recites “the second shelf” and from the context of Applicant’s specification, Examiner believes that Applicant means that “the second shelf” is the shelf behind the robot and not the shelf that it is photographing. Claim 2 limits Claim 1 to shelves facing each other and shelves claimed in Claim 12, without this limitation, could mean any shelves. Williams does not teach that a robot is in communication with the electronic shelf label “on the second shelf” *with the additional limitation that the second shelf label is behind the robot*. After a diligent search Examiner is not able to find art that teaches this particular combination of elements. Examiner holds that, depending on specific amended claim construction, a claim that is a combination of Claims 1, 2 and 12 could be allowable over prior art.
Regarding Claim 13:
The robot of claim 1, wherein the controller is further configured to: control the driving module to move the body between the first shelf and the second shelf facing each other among the plurality of shelves to a first position closer to a second shelf than to the first shelf, control the imaging device to capture a first image of the first shelf at the first position, control the electronic shelf identification device to obtain, at the first position, identification information of a first electronic shelf label adjacent to the first position among a plurality of electronic shelf labels provided on the second shelf, control the driving module to move the body between the first shelf and the second shelf to a second position closer to the first shelf than to the second shelf, control the imaging device to capture a second image of the second shelf at the second position, and control the electronic shelf identification device to obtain, at the second position, identification information of a second electronic shelf label adjacent to the second position among a plurality of electronic shelf labels provided on the first shelf.
Claim 13 recites “between the first shelf and the second shelf facing each other”, but, as discussed in the 35 USC 112b rejection, above, these elements are indefinite because they lack proper antecedent basis. If these elements are amended to correct this rejection and first and second shelf are correctly claimed, Examiner holds that Williams does not teach that a robot is in communication with the electronic shelf label “on the second shelf” *with the additional limitation that the second shelf label is behind the robot*. After a diligent search Examiner is not able to find art that teaches this particular combination of elements. Examiner holds that, depending on specific amended claim construction, a claim that is a combination of Claims 1 and 13 could be allowable over prior art.
Regarding Claim 14:
The robot of claim 13, wherein the controller is further configured to: generate first realogram data including display status of products provided on the first shelf based on the first image, and generate second realogram data including display status of products provided on the second shelf based on the second image.
As currently claimed, Claim 14 is dependent on Claim 13. Examiner holds that if Claim 13 is amended as recommended above and is allowable, Claim 14, if rolled up into the combination and properly constructed, could also be allowable.
Regarding Claim 15:
The robot of claim 14, wherein, in generating the first realogram data, the controller is configured to include product-related information assigned to the second electronic shelf label corresponding to the identification information of the second electronic shelf label obtained by the electronic shelf label identification device in the first realogram data, and in generating the second realogram data, the controller is configured to include product-related information assigned to the first electronic shelf label corresponding to the identification information of the first electronic shelf label obtained by the electronic shelf label identification device in the second realogram data.
As currently claimed, Claim 15 is dependent on Claim 14 which is dependent on Claim 13. Examiner holds that if Claim 13 is amended as recommended above and is allowable, and Claim 14 is amended as recommended above and is allowable, that Claim 15, if rolled up into the combination and properly constructed, could also be allowable.
Allowable Subject Matter
Claim 6 and 16-17 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
Regarding Claim 6.
The robot of claim 3, wherein, in obtaining the information about the first position, the controller is configured to control the electronic shelf label identification device to obtain, at the first position, identification information of a first electronic shelf label adjacent to the first position among a plurality of electronic shelf labels provided on the second shelf and obtain, as the information about the first position, position information of the first electronic shelf label based on the identification information of the first electronic shelf label; and in obtaining the information about the second position, the controller is configured to control the electronic shelf label identification device to obtain, at the second position, identification information of a second electronic shelf label adjacent to the second position among a plurality of electronic shelf labels provided on the second shelf and obtain, as the information about the second position, position information of the second electronic shelf label based on the identification information of the second electronic shelf label.
While Williams teaches a robot moving along an aisle closer to one shelf than another (see [0063] and [Fig 2]), teaches obtaining (robot) position information (see [0058] and [0061]), teaches the robot reading shelf labels (see at least [0073]) and teaches the robot obtaining information electronically from electronic shelf labels ([0139] “the robot or movable platform could come into proximity with an electronic device, such as an electronic shelf label (ESL) or smart shelf. The robot or movable platform can charge these electronic devices inductively or via RFID, collect data wirelessly from these electronic devices”), Williams does not teach obtaining (robot) position information electronically from the electronic shelf labels behind them. After a diligent search Examiner is not able to find art that teaches this particular combination of elements. Further, as discussed in the 35 USC 112b rejection, above, Applicant’s use of “the first shelf” and “the second shelf” are indefinite and are rejected for insufficient antecedent basis. Examiner holds that, depending on specific amended claim construction, a claim that is a combination of Claims 1, 2, 3 and 6 could be allowable over prior art.
Regarding Claim 16:
The robot of claim 13, wherein the controller is further configured to: control the driving module to move the body to position 1-1 closer to a second shelf than to the first shelf, control the imaging device to obtain, at the position 1-1, a first target image of the first electronic shelf label, which is one of a plurality of electronic shelf labels provided on the second shelf, control the driving module to move the body to position 1-2 closer to the second shelf than the position 1-1 and adjacent to the first electronic shelf label, control the imaging device to capture the first image of the first shelf at the position 1-2, and control the electronic shelf label identification device to obtain identification information of the first electronic shelf label at the position 1-2.
While Williams teaches taking a picture of a shelf label (see at least [0073]), teaches taking wide angle images of shelves and close-in images of shelf labels (see at least [0059]), Williams does not specifically teach moving closer to the shelf label (closer to the second shelf than the position 1-1) and then taking a picture across the aisle (capture the first image of the first shelf) and does not teach getting shelf tag information from the shelf tag behind them. Williams does not teach that a robot is in communication with the electronic shelf label “on the second shelf”, meaning the shelf label behind the robot. After a diligent search Examiner is not able to find art that teaches this particular combination of elements. Examiner holds that, depending on specific amended claim construction, a claim that is a combination of Claims 1, 13 and 16 could be allowable over prior art.
Regarding Claim 17:
The robot of claim 16, wherein, while generating realogram data including display status of products provided on the second shelf based on the second image, the controller is configured to: specify the first electronic shelf label from the second image by comparing the second image with a first target image of the first electronic shelf label provided on the second shelf, specify a product adjacent to the first electronic shelf label from the second image, and match product-related information assigned to the first electronic shelf label extracted based on the identification information of the first electronic shelf label to a product adjacent to the first electronic shelf label specified in the second image.
Williams teaches this- Williams teaches taking a picture of a shelf label (see at least [0073]), teaches taking wide angle images of shelves and close-in images of shelf labels (see at least [0059]), teaches matching shelf labels to products (see at least [0063] and [0093]). Because Claim 17 is dependent on Claim 16 which could be allowable if amended as suggested above, Claim 17 could also be allowable.
Relevant Prior Art Not Relied Upon
The prior art is made of record and not relied upon is considered pertinent to applicant’s disclosure. The additional cited art further establishes the state of the art at the time of applicant’s application.
U.S. Patent 10,007,964 (Calhoon) teaches using electronic shelf labels as beacons (Column 5, line 63 to Column 7, line 3), teaches that electronic shelf labels can trigger a robot to take an action (Column 7, lines 3-19), and teaches that an electronic shelf label can provide planogram location information using a remote database (Column 12, lines 24-34).
U.S. Patent Publication 2019/0215424 (Adato) teaches using robots with cameras to obtain inventory information in a retail store and teaches generating a realogram and a database linking location and identity of products.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KIMBERLY S BURSUM whose telephone number is (571)272-8213. The examiner can normally be reached M-F 9:30 AM - 6:30 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Florian (Ryan) m Zeender can be reached at 571-272-6790. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KIMBERLY S. BURSUM/Examiner, Art Unit 3627