DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-22 have been considered but are not persuasive.
In response to Applicant’s argument concerning the TC55 reference, the Examiner disagrees. The camera is located in the same location as most cell phones, while the line scanner is located on the top edge. In the video 0:22 shows the linear imager and 0:18 shows the 8MP Camera
PNG
media_image1.png
472
844
media_image1.png
Greyscale
PNG
media_image2.png
454
848
media_image2.png
Greyscale
Upon further inspection, the division of Motorola that makes the TC55, was bought by a company called Zebra Technologies1 in October 2014. The manual for the TC-55, discontinued in 2017, can be found here https://topresale.ru/storage/tiny/resheniya/instructions/zebra_motorola_symbol_tc55_user_guide.pdf
And shows exactly what the video shows :
PNG
media_image3.png
348
472
media_image3.png
Greyscale
PNG
media_image4.png
411
584
media_image4.png
Greyscale
PNG
media_image5.png
70
582
media_image5.png
Greyscale
In regards to Applicant’s argument concerning Tajeddin, Applicant’s arguments are moot in view of new grounds of rejection.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1,3, 7-8, 12, and 18-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Plummer et al. (US 20190156086 A1), hereinafter Plummer in view of TCC55(https://www.youtube.com/watch?v=HSP_4CIDHDw; “Motorola TC55 Introduction mobile computer scanner barcode scanner”, April 20, 2015) in view of Khawand (PGPub 2013/0057713), hereafter Khawand.
Regarding claim 1, Plummer teaches a mobile device, comprising:
a camera (Plummer, para. [0117]: “
PNG
media_image6.png
595
622
media_image6.png
Greyscale
”);
a display (Plummer, para. [0084]: “
PNG
media_image7.png
421
621
media_image7.png
Greyscale
”);
a tracking sensor (Plummer, para. [0135]: “
PNG
media_image8.png
177
622
media_image8.png
Greyscale
PNG
media_image9.png
251
616
media_image9.png
Greyscale
”; where a tracking sensor is understood to mean a sensor that is used in tracking, and one of the image sensors of the mobile is used as a camera while another sensor is used as a tracking sensor); and
a controller connected to a repository of item identifiers and item positions in a facility frame of reference for items disposed on support surfaces within a facility (Plummer, fig. 1A; para. [0094], lines 1-14: “
PNG
media_image10.png
236
625
media_image10.png
Greyscale
PNG
media_image11.png
168
619
media_image11.png
Greyscale
PNG
media_image12.png
923
1114
media_image12.png
Greyscale
”), the controller configured to:
track, via the tracking sensor, successive poses of the mobile device in the facility frame of reference (Plummer, para. [0094], lines 15-26: “
PNG
media_image13.png
345
619
media_image13.png
Greyscale
”);
control the camera to capture a stream of images while tracking the poses, and for each image (Plummer, para. [0092], lines 1-12: “
PNG
media_image14.png
258
618
media_image14.png
Greyscale
PNG
media_image15.png
93
624
media_image15.png
Greyscale
”):
determine, based on the tracked poses, whether to perform item detection (Plummer, para. [0092], lines 10-24: “
PNG
media_image16.png
423
623
media_image16.png
Greyscale
”), and
when the determination is affirmative, (i) use a sensor(Plummer, para. [0066]
PNG
media_image17.png
120
420
media_image17.png
Greyscale
, scanning the bar code) (ii) decode item identifiers from the indicia (Plummer, para. [0093], lines 1-18: “
PNG
media_image18.png
511
618
media_image18.png
Greyscale
”), (iii) generate positions of the detected indicia in the facility frame of reference, based on the poses of the mobile device (Plummer, para. [0093], lines 1-18 (see above); see lines 17-18 specifically), and (iv) update the repository with the decoded item identifiers and the generated positions (Plummer, para. [0081], lines 1-8; [0094], lines 55-60: “
PNG
media_image19.png
167
619
media_image19.png
Greyscale
PNG
media_image20.png
224
620
media_image20.png
Greyscale
;
”).
Plummer (Paragraph 66) discloses “In alternative embodiments, the container may include a simple identification means, such as a bar code. The movement and location of the container or object may be tracked by (1) first identifying the container or object by manual input of the identification code and/or scanning or capturing an image having a field of view noncollinear with the field of view of the camera to allow the image processing host to identify the specific container or object”, Thus discloses a sensor, such as a bar code sensor, to detect indicia affixed to the items. But does not explicitly disclose the orientation of the sensor, in particular “use a sensor having a centerline of a field of view at a non-zero angle with respect to a centerline of a field of view of the camera to
” (TC55, minutes 0:15 and 0:19, shows camera sensor and barcode reader with different fields of view
PNG
media_image21.png
670
1212
media_image21.png
Greyscale
PNG
media_image22.png
678
1132
media_image22.png
Greyscale
)
Examiner Note: At the store https://www.barcodesinc.com/, there are numerous barcode scanners + cameras devices for sale, which have scanner and camera having different fields of view.
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to use the device of TC55 in order to capture the images and read the bar codes used in Plummer.
The suggestion/motivation for doing so would have been it provides a convenient device which includes both the camera and scanner in one unit, which Plummer requires but fails to explicitly disclose.
Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Plummer discloses determining based on tracked poses whether to perform item detection, but Plummer does not expressly disclose “comparing device motion indicated by the tracked poses to a motion threshold and initiating the item detection when the device motion is below the motion threshold,”
Khawand teaches “comparing device motion indicated by the tracked poses to a motion threshold and initiating the item detection when the device motion is below the motion threshold,2” (Khawand, Fig. 7
PNG
media_image23.png
1166
530
media_image23.png
Greyscale
PNG
media_image24.png
254
610
media_image24.png
Greyscale
PNG
media_image25.png
118
606
media_image25.png
Greyscale
)
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to use the determined motion and threshold of Khawand to determine when to acquire images of Plummer.
The suggestion/motivation for doing so would have been to reduce Blur in the images. When the pose of camera is moving it will cause blurring images during acquisition.
Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Plummer with Khawand and TC55 to obtain the invention as specified in claim 1.
Regarding claim 3, Plummer in view of TC55 in view of Khawand teaches the mobile device of claim 1, wherein the controller is further configured to:
receive a task definition containing one of the item identifiers (Plummer, para. [0066], lines 1-7: “
PNG
media_image26.png
202
617
media_image26.png
Greyscale
”; where the task definition is tracking and recording the movement and position of items and the item identifier is the matrix code);
determine whether an item position corresponding to the one of the item identifiers is in the visible subset (Plummer, para. [0097], lines 1-7 and 18-25 (see claim 2 rejection); understanding the limitation to mean determining if an item location corresponding to an item identifiers is in the repository); and
when the item position corresponding to the one of the item identifiers is in the visible subset, highlight the corresponding one of the indicators in an overlay to the image (Plummer, para. [0097] (see claim 2 rejection); para. [0088]: “
PNG
media_image27.png
539
615
media_image27.png
Greyscale
”; where the indicators are locations and directions to the items, and the highlighted indicators are the arrows that show the direction of travel).
Regarding claim 7, Plummer in view of TC55 in view of Khawand teaches the mobile device of claim 1, wherein the controller is configured to update the repository by transmitting the decoded item identifiers and the generated positions to a server hosting the repository (Plummer, para. [0081], lines 1-8; para. [0094], lines 55-67; para. [0117], lines 1-5: “
PNG
media_image28.png
225
618
media_image28.png
Greyscale
;
PNG
media_image29.png
366
624
media_image29.png
Greyscale
;
PNG
media_image30.png
145
616
media_image30.png
Greyscale
”).
Regarding claim 8, Plummer in view of TC55 in view of Khawand teaches the mobile device of claim 1, further comprising a memory storing the repository (Plummer, para. [0082]: “
PNG
media_image31.png
623
621
media_image31.png
Greyscale
”).
With regards to claims 12,14 and 18-19, they recite the functions of the mobile device of claims 1-3 and 7-8 as a method. Plummer discloses “system and methods for tracking items” (Plummer, abstract). Regarding the remaining limitations of claims 12,14 and 18-19, the analysis in rejecting claims 1,3 and 7-8 is equally applicable to claims 12,14 and 18-19.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 4-5 9-10, 15-16 and 20-21 are rejected under 35 U.S.C. 103 as being unpatentable over Plummer in view of TC55 in view of Khawand in view of Tajeddin et al. (US 20200182622 A1), hereinafter Tajeddin.
Regarding claim 4, Plummer in view of TC55 in view of Khawand teaches the mobile device of claim 1.
Plummer fails to teach wherein the item identifiers and item positions of the repository include initial item identifiers and item positions detected by a mobile automation apparatus.
However, Tajeddin teaches wherein the item identifiers and item positions of the repository include initial item identifiers and item positions detected by a mobile automation apparatus (Tajeddin, fig. 1; fig. 2B; para. [0026]; para. [0029]: “
PNG
media_image32.png
287
619
media_image32.png
Greyscale
PNG
media_image33.png
593
504
media_image33.png
Greyscale
PNG
media_image34.png
286
625
media_image34.png
Greyscale
PNG
media_image35.png
454
342
media_image35.png
Greyscale
”; where the shelf data includes the location of the shelf and images of shelf products, and include initial item identifiers and item positions since all identifiers and positions are found using the mobile apparatus).
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to add “the item identifiers and item positions of the repository include initial item identifiers and item positions detected by a mobile automation apparatus“ as disclosed by Tajeddin into the inventory guidance and mapping system of Plummer.
The suggestion/motivation for doing so would have been the benefits of automating a process, which include reduced human error, faster production, and lower operating costs.
Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Tajeddin with Plummer to obtain the invention as specified in claim 4.
Regarding claim 5, Plummer in view of TC55 in view of Khawand teaches the mobile device of claim 1.
Plummer fails to teach wherein the tracking sensor includes at least one of an inertial measurement unit (IMU) and a depth sensor.
However, Tajeddin teaches wherein the tracking sensor includes at least one of an inertial measurement unit (IMU) and a depth sensor (Tajeddin, para. [0034]: “
PNG
media_image36.png
281
620
media_image36.png
Greyscale
PNG
media_image37.png
169
619
media_image37.png
Greyscale
”).
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to add “the tracking sensor includes at least one of an inertial measurement unit (IMU) and a depth sensor” as disclosed by Tajeddin into the inventory guidance and mapping system of Plummer.
The suggestion/motivation for doing so would have been to accurately measure distance information in an enclosed space.
Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Tajeddin with Plummer to obtain the invention as specified in claim 5.
Regarding claim 9, Plummer in view of TC55 in view of Khawand teaches the mobile device of claim 8, wherein the controller is further configured to:
store, with each decoded item identifier and generated position, a timestamp indicating when the decoded item identifier was most recently detected at the generated position (Plummer, para. [0081], lines 1-8 (see claim 7 rejection).
Plummer fails to teach:
assign a confidence level to the decoded item identifier; and
periodically updating the confidence level based on an age of the decoded item identifier.
However, Tajeddin teaches:
assign a confidence level to the decoded item identifier (Tajeddin, abstract: “
PNG
media_image38.png
512
624
media_image38.png
Greyscale
”; where the decoded item identifier is the localization); and
periodically updating the confidence level based on an age of the decoded item identifier Tajeddin, abstract (see above); where the age is the set amount of time between each repetition).
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to add “assign a confidence level to the decoded item identifier; and periodically updating the confidence level based on an age of the decoded item identifier” as disclosed by Tajeddin into the inventory guidance and mapping system of Plummer.
The suggestion/motivation for doing so would have been to increase the accuracy of the mobile device (Tajeddin, abstract).
Therefore, it would have been obvious to combine Tajeddin with Plummer in view of TC55 to obtain the invention as specified in claim 9.
Regarding claim 10, Plummer in view of TC55 in view of Khawand in view of Tajeddin teaches the mobile device of claim 9, wherein the controller is configured to track the successive poses by:
retrieving at least a portion of the repository (Plummer, para. [0081], lines 1-8 (see claim 7 rejection)).
Plummer fails to teach selecting an anchor subset of item identifiers and associated positions, having confidence levels above a threshold.
However, Tajeddin teaches selecting an anchor subset of item identifiers and associated positions, having confidence levels above a threshold (Tajeddin, fig. 5B; para. [0053]: “
PNG
media_image39.png
311
623
media_image39.png
Greyscale
PNG
media_image40.png
357
499
media_image40.png
Greyscale
”).
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to add “selecting an anchor subset of item identifiers and associated positions, having confidence levels above a threshold” as disclosed by Tajeddin into the inventory guidance and mapping system of Plummer.
The suggestion/motivation for doing so would have been to increase the accuracy of the mobile device (Tajeddin, abstract).
Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Tajeddin with Plummer to obtain the invention as specified in claim 10.
With regards to claims 15-16 and 20-21, they recite the functions of the mobile device of claims 4-5 and 9-10 as a method. Plummer discloses “system and methods for tracking items” (Plummer, abstract). Regarding the remaining limitations of claims 15-17 and 20-21, the analysis in rejecting claims 4-5 and 9-10 is equally applicable to claims 15-16 and 20-21.
Claims 11 and 22 are rejected under 35 U.S.C. 103 as being unpatentable over Plummer in view of TC55 in view of Khawand , in view of Stenning et al. (US 20180328753 A1), hereinafter Stenning.
Regarding claim 11, Plummer in view of TC55 teaches the mobile device of claim 1, wherein the controller is configured to generate the positions of the detected indicia in the facility frame of reference, by:
storing timestamps in association with [each tracked pose and] each of the images (Plummer, para. [0081], lines 1-8 (see claim 7 rejection); where the timestamp is the time and date information).
Plummer fails to teach:
storing timestamps in association with each tracked pose and each of the images; and
responsive to detecting the indicia in the image, retrieving a tracked pose having a timestamp matching the image timestamp.
However, Stenning teaches:
storing timestamps in association with each tracked pose and each of the images (Stenning, fig. 4A; para. [0011]; para. [0084]; para. [0085], lines 1-6: “
PNG
media_image41.png
223
623
media_image41.png
Greyscale
PNG
media_image42.png
170
622
media_image42.png
Greyscale
PNG
media_image43.png
450
623
media_image43.png
Greyscale
PNG
media_image44.png
647
415
media_image44.png
Greyscale
”; where para. [0084] discusses how each pose is timestamped and para. [0085] discusses how periodic sensor data (like that from a camera in para. [0011]) are also timestamped); and
responsive to detecting the indicia in the image, retrieving a tracked pose having a timestamp matching the image timestamp (Stenning, para. [0084] (see above); fig. 8; para. [0099]: “
PNG
media_image45.png
661
495
media_image45.png
Greyscale
PNG
media_image46.png
915
489
media_image46.png
Greyscale
”; where the landmarks are visual features that are equivalent to indicia).
Plummer includes timestamps with image taken which is associated, but does not explicitly teach having timestamps associated with the tracked poses. It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to add “storing timestamps in association with each tracked pose and each of the images; and responsive to detecting the indicia in the image, retrieving a tracked pose having a timestamp matching the image timestamp” as disclosed by Stenning into the inventory guidance and mapping system of Plummer.
The suggestion/motivation for doing so would have been to identify features on the travel path of the mobile device which can then be used to constrain the created map of the environment (Stenning, abstract).
Therefore, it would have been obvious to combine Stenning with Plummer in view of TC55 to obtain the invention as specified in claim 11.
Plummer in view of TC55 in view of Khawand in view of Stenning teaches:
generating the positions of the detected indicia using the retrieved tracked pose (Plummer, para. [0093]).
With regards to claim 22, it recites the functions of the mobile device of claim 11 as a method. Plummer discloses “system and methods for tracking items” (Plummer, abstract). Regarding the remaining limitations of claim 22, the analysis in rejecting claim 11 is equally applicable to claim 22.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GANDHI THIRUGNANAM whose telephone number is (571)270-3261. The examiner can normally be reached M-F 8:30-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached at 571-272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GANDHI THIRUGNANAM/Primary Examiner, Art Unit 2672
1 Same name as the pending Applicant.
2 This claim is supported by Paragraph 45 and 34 of the disclosure as filed.