Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments have been considered. See the rejection below incorporating Tang (20210383594) which teaches an work spot and a bounding box in pars. 35-38 and 43-46.
Regarding new claims 16-17, please see the rejection of claims 1, 9 and 13.
Regarding new claims 14-15 and 18-20, these are objected to below.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-11 and 16-17 are rejected under 35 U.S.C. 103 as being unpatentable over Datar (20210124944) in view of Zia (11216656) in further view of Tang (20210383594).
.
Regarding claim 1, Datar teaches a processing system, the system estimates a pose of a worker (par. 437-438),
a position of an article, an orientation of the article, and a state of the article based on an image of the worker and the article (fig. 41 and pars. 475-477. Determining the interaction between the person and the object includes finding the position, orientation and state of the item on the shelf as to whether is it being picked up by the user),
estimates a work spot of the worker on the article based on an estimation result of the pose (par. 474, position of person) ,
an estimation result of the position (par. 474), and
an estimation result of the orientation, and estimates a task performed by the worker based on an estimation result of the work spot and an estimation result of the state (par. 478, determining if the user is picking up the object).
Zia teaches refer to a task database including a plurality of datasets, each of the plurality of datasets including a task, a state of the article in the task, and a work spot of the worker in the task: and estimate a task performed by the worker by comparing estimation results of the work spot and the state with the plurality of datasets (see col. 10 line 62 to col. 12 line 35).
It would have been obvious prior to the effective filing date of the invention to one of ordinary skill in the art to include in Datar the ability to validate tasks as taught by Zia. The reason is to check the worker against tasks.
Tang teaches wherein estimating the work spot includes: setting a plurality of gates on the article based on the estimated position and the estimated orientation (see par. 43 and 45, ROI using position and orientation),
the plurality of gates corresponding to mutually separated regions on a surface of the article (see pars. 45, control points on any point on the surface making bounding box),
and estimating the work spot based on an intersection relationship between either a line segment indicating a body part of the worker or an extension of the line segment, and at least one of the plurality of gates (par. 43 and 46, the ray object intersection),
the body part at least including a lower arm of the worker (pars. 35-38, based on the joint of the user’s arm, extend ray. This includes an elbow).
It would have been obvious prior to the effective filing date of the invention to one of ordinary skill in the art to include in Datar and Zia the ability to setting a region around an object with a work spot as taught by Tang. The reason is to allow the system to focus on a specific region.
Regarding claim 2, wherein the estimation of the position and the orientation includes: extracting a partial region from the image, the partial region including the worker and the article (see figure 42 item 4204 and par. 494, ROI);
specifying a worker region based on the estimation result of the pose, the worker being imaged in the worker region (par. 494 the area where the interaction is happening based on the orientation of the user and the object);
estimating an article region by removing the worker region from the partial region, the article being imaged in the article region; and estimating the position and the orientation by using the article region (par. 494-495, this ROI is focusing on the object and the wrist and does not include the rest of the person).
Regarding claim 3, see par. 301 contour detection of the object.
Regarding claim 4, see par. 494.
Regarding claim 5, see pars. 573-574, tracking multiple objects on a shelf.
Regarding claim 6, see par. 494-495.
Regarding claim 7, see par. 301 which discusses contour lines to detect an object and see par. 494-495 which uses the pose based on the skeleton of the user and its interaction with the object. Gates is read as the outline of the object. Also see table 1 of Zia in col. 10 lines 40-65, did a hand touch a specific point.
Regarding claim 8, see the abstract and the rejection of claim 1 where this process is tracked through multiple frames.
Regarding claim 9, see par. 296, using a time threshold.
Regarding claims 10-11, see the rejection of claim 1.
Regarding claim 12, see the rejection of claim 7, see table 1 of Zia in column 10 which discusses touch, interacting and matching uses interactions and to determine the work area, like workers holding two sides of an object. Touch determines the object location.
Regarding claim 13, see col. 10 line 62 to col. 12 line 35 of Zia.
Regarding new claims 16-17, please see the rejection of claims 1, 9 and 13.
Allowable Subject Matter
Claims 14-15 and 18-20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
These narrowing details like wrist elbow, directions, window sizing and ratio thresholds are not found in the prior art.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HADI AKHAVANNIK whose telephone number is (571)272-8622. The examiner can normally be reached 9 AM - 5 PM Monday to Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Henok Shiferaw can be reached at (571) 272-4637. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HADI AKHAVANNIK/Primary Examiner, Art Unit 2676