Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
The following is a quotation of pre-AIA 35 U.S.C. 112, fourth paragraph:
Subject to the following paragraph [i.e., the fifth paragraph of pre-AIA 35 U.S.C. 112], a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
Claim 11 is objected to because it is missing the reference to the claim from which it depends. “11. The method of claim ??, ------.”. Amendments/clarification are required.
Claim 12 is objected to because it is missing the reference to the claim from which it depends. “12. The method of claim ??, ------.”. Amendments/clarification are required.
Claim 13 is objected to because it is missing the reference to the claim from which it depends. “13. The method of claim ??, ------.”. Amendments/clarification are required.
Claim Interpretation
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “an acquisition module, a key point detection module and a relative pose determination module” in claim 14.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 6 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Patel et al., (US20150117788) hereafter Patel in view of Huang Hong et al., (CN106556341B) hereafter Huang.
1. Regarding claim 1, Patel discloses a shelf positioning method (figs 2-7 and 9 shows and discloses a shelf positioning method), comprising:
acquiring image data, in an environment where a movable equipment is located, through an image acquisition module in the movable equipment (fig 2 and paras 0021, 0023 shows and discloses acquiring image data, in an environment where a movable equipment is located, through an image acquisition module in the movable equipment);
inputting the image data into a key point detection network to extract first position information of shelf key points in an image coordinate system from the image data through the key point detection network (paras 0022-0025 shows and discloses shelf edge detection component 304 detecting shelf edge corners 410 located in the coordinates (image coordinates) from the captured/acquired image 400 using a geometric model fitting and verification algorithm (i.e key point detection network) to estimate a shape based on the detected interest point matches meeting the above claim limitations);
Huang discloses determining a relative pose of the shelf key points relative to the movable equipment according to the first position information and a conversion relationship between the image coordinate system and a vehicle body coordinate system of the movable equipment (Abstract and the disclosure pages 2-3 discloses “The invention claims a shelf position (pose) deviation detection method and system based on feature information image, step one, mounted on the video camera on the robot, step two, the mapping relationship (i.e the conversion relationship) between the calibration camera pixel coordinate system and the robot coordinate system (i.e the vehicle body coordinate system); step three, the shelf bottom set pattern having characteristic information of measured image characteristic point coordinate (i.e image coordinate system) in the shelf coordinate system, step four, the pixel coordinate robot lifting shelf rear view camera, scanning image, obtaining the image feature point, step five, mapping the pixel coordinates in the step of mapping relation, calculating the image feature point coordinates in the robot coordinate system; step six, calculating shelf deviation (i.e change relative to the first position) relative to the pose of the robot. The invention is a camera detecting the position deviation of the shelf (i.e relative to the first position of the shelf (i.e shelf corner key points) meeting the above claim limitations, examiner notes that the specifics of the image coordinate system, a vehicle body coordinate system and a conversion relationship are not required by the current claim). Before the effective filing date of the invention was made, Patel and Huang are combinable because they are from the same filed of endeavor and are analogous art of image processing. The suggestion/motivation would be a convenient, fast and low cost method/system on page 3. Therefore, it would be obvious and within one of ordinary skill in the art to have recognized the advantages of Huang in the method of Patel to obtain the invention as specified in claim 1.
2. Regarding claim 6, Patel and Huang disclose the method of claim 1, wherein the shelf key points comprise any one or more of a top left vertex, a top right vertex, a bottom left vertex and a bottom right vertex of a shelf (fig 4
PNG
media_image1.png
127
494
media_image1.png
Greyscale
shows one or more of a top left vertex, a top right vertex, a bottom left vertex and a bottom right vertex 410 of a shelf 402 meeting the claim limitations).
3. Claim 14 is a corresponding device claim of claim 1. See the corresponding explanation of claim 1. Patel shows and discloses a shelf positioning device in Figs 2-4.
Claims 8, 10 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Patel in view of Huang and in further view of Dhayalkar et al. (US20220042824) hereafter Dhayalkar.
4. A shelf connecting (figs 2-7 and 9 shows and discloses a shelf connecting method) method, comprising:
acquiring image data, in an environment where a movable equipment is located, through an image acquisition module in the movable equipment (fig 2 and paras 0021, 0023 shows and discloses acquiring image data, in an environment where a movable equipment is located, through an image acquisition module in the movable equipment);
inputting the image data into a key point detection network to extract first position information of shelf key points in an image coordinate system from the image data through the key point detection network (paras 0022-0025 shows and discloses shelf edge detection component 304 detecting shelf edge corners 410 located in the coordinates (image coordinates) from the captured/acquired image 400 using a geometric model fitting and verification algorithm (i.e key point detection network) to estimate a shape based on the detected interest point matches meeting the above claim limitations);
and determining a connecting route between the movable equipment and a shelf according to the relative pose, so that the movable equipment executes a connecting process based on the connecting route.
Huang discloses determining a relative pose of the shelf key points relative to the movable equipment according to the first position information and a conversion relationship between the image coordinate system and a vehicle body coordinate system of the movable equipment (Abstract and the disclosure pages 2-3 discloses “The invention claims a shelf position (pose) deviation detection method and system based on feature information image, step one, mounted on the video camera on the robot, step two, the mapping relationship (i.e the conversion relationship) between the calibration camera pixel coordinate system and the robot coordinate system (i.e the vehicle body coordinate system); step three, the shelf bottom set pattern having characteristic information of measured image characteristic point coordinate (image coordinate system) in the shelf coordinate system, step four, the pixel coordinate robot lifting shelf rear view camera, scanning image, obtaining the image feature point, step five, mapping the pixel coordinates in the step of mapping relation, calculating the image feature point coordinates in the robot coordinate system; step six, calculating shelf deviation (i.e relative to the first position) relative to the pose of the robot. The invention is a camera detecting the position deviation of the shelf (i.e relative to the first position of the shelf (i.e shelf corner key points) meeting the above claim limitations, examiner notes that the specifics of the image coordinate system, a vehicle body coordinate system and a conversion relationship are not required by the current claim)
. Before the effective filing date of the invention was made, Patel and Huang are combinable because they are from the same filed of endeavor and are analogous art of image processing. The suggestion/motivation would be a convenient, fast and low-cost method/system on page 3. Patel and Huang both disclose capturing image of the shelf (key points) using the movable camera. Patel and Huang are however silent and fails to disclose determining a connecting route between the movable equipment and a shelf according to the relative pose, so that the movable equipment executes a connecting process based on the connecting route.
Dhayalkar shows and discloses and determining a connecting route between the movable equipment and a shelf according to the relative pose, so that the movable equipment executes a connecting process based on the connecting route (Figs 2-3, paras 0029-0030, 0054 and 0067 shows and discloses and determining a connecting route between the movable equipment and a shelf according to the relative pose, so that the movable equipment executes a connecting process based on the connecting route 302). Before the effective filing date of the invention was made, Dhayalkar, Patel and Huang are combinable because they are from the same filed of endeavor and are analogous art of sensor data processing. The suggestion/motivation would be an advantageous and improved system/method (para 0039). Therefore, it would be obvious and within one of ordinary skill in the art to have recognized the advantages of Huang (i.e a convenient, fast and low-cost method/system on page 3) and an advantageous and improved system/method (para 0039) as disclosed by Dhayalkar to obtain the invention as specified in claim 8.
5. Regarding claim 10, Patel, Huang and Dhayalkar discloses the method of claim 8. Patel discloses further comprising: navigating the movable equipment to a connecting position matched with the shelf to be connected, the connecting position being a position for collecting at least one shelf key point in the environment where the movable equipment is located; wherein acquiring image data in the environment where a movable equipment is located through an image acquisition module in the movable equipment comprises: photographing the image data, in the environment where the movable equipment is located, through the image acquisition module in the connecting position (para 0021 discloses the camera 210 mounted on the movable apparatus that is moved along aisle 202 to capture (photograph images) of the shelf and Fig 4 shows
PNG
media_image1.png
127
494
media_image1.png
Greyscale
that the movable camera 210 can navigate and capture at least shelf keypoints at the corresponding matched positions meeting the claim limitations).
6. Regarding claim 12 as best understood by the examiner see the explanation of claim 1. Examiner notes that from the teachings of Dhayalkar, correcting the connecting process and re-acquiring the image data would be obvious and also re-executing the relative pose determination would also be obvious in view of the teachings of Huang.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Patel in view of Huang, Dhayalkar and in further view of NPL8 (Robust Fusion of LiDAR and Wide-Angle Camera Data for Autonomous Mobile Robots, Varuna De Silva et al., MDPI, 2018, Pages 1-21) hereafter NPL8.
7. Regarding claim 13 as best understood by the examiner, Pate, Huang and Dhayalkar disclose the method of claim 8. Patel discloses wherein the image acquisition module camera 210 in para 0021. Patel, Huang and Dhayalkar however are silent and fail to disclose wherein the image acquisition module is a wide-angle monocular camera.
NPL8 discloses wherein the image acquisition module is a wide-angle monocular camera (title, abstract and Pages 2-3, Fig 1 (a), fig 2. disclose wherein the image acquisition module is a wide-angle monocular camera). Before the effective filing date of the invention was made, Patel, Huang, Dhayalkar and NPL8 are combinable because they are from the same field of endeavor and are analogous art of image processing. The suggestion/motivation would be a compact and cost-effective system (Page 2). Therefore, it would be obvious and within one of ordinary skill in the art to have recognized the advantages of NPL 8 in the method of Patel, Huang and Dhayalkar to obtain the invention as specified in claim 13.
Examiner's Note: Examiner has cited figures, and paragraphs in the references as applied to the claims above for the convenience of the applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested for the applicant, in preparing the responses, to fully consider the references in entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the examiner. Examiner has also cited references in PTO892 but not relied on, which are relevant and pertinent to the applicant’s disclosure, and may also be reading (anticipatory/obvious) on the claims and claimed limitations. Applicant is advised to consider the references in preparing the response/amendments in-order to expedite the prosecution.
Allowable Subject Matter
Claims 2-5, 7, 9 and 18-21 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim 11 would be allowable if rewritten to overcome the objections under 35 U.S.C. 112(d) or 35 U.S.C. 112 (pre-AIA ), 4th paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JAYESH PATEL whose telephone number is (571)270-1227. The examiner can normally be reached IFW Mon-FRI.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached at 571-270-5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
JAYESH PATEL
Primary Examiner
Art Unit 2677
/JAYESH A PATEL/Primary Examiner, Art Unit 2677