DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-18 are pending in the application. Claims 2 and 5 have been amended.
The amendment filed 1/30/26 overcomes objections to claims 2 and 5.
Response to Arguments
Applicant's arguments filed 1/30/26 have been fully considered but they are not persuasive. Response to the arguments is presented below.
Arguments (REMARKS pages 1-3)
In the REMARKS, Applicant asserted that Liao or Liao in combination with KIMURA does not teach: in claim 9 “when the quality indicator does not satisfy the predetermined threshold, generate a feedback notification”, in claim 1 “when the quality indicator does not satisfy the predetermined threshold, generating a positional notification”, and in claim 18 “selecting, based on the quality indicator, between (i) generating feedback to reposition a sensor relative to the object and (ii) obtaining dimensions of the object from the point cloud”. Applicant further cited the instant specification to give a description of what “a notification” is, such as para. [0035], and concluded that Liao “describes an internal logic branch (Proceed/Abort), not a "notification" generated for a user or external entity”. Examiner respectfully disagrees.
During patent examination, the pending claims must be "given their broadest reasonable interpretation consistent with the specification." (MPEP §2111). "Though understanding the claim language may be aided by explanations contained in the written description, it is important not to import into a claim limitations that are not part of the claim. For example, a particular embodiment appearing in the written description may not be read into a claim when the claim language is broader than the embodiment." (MPEP §2111.01 II). In current application, under broadest reasonable interpretation, “a notification” (or “feedback”) is not necessarily a notification/feedback for a user or external entity, such as those described in instant para.[0035]. Examiner’s interpretation of “a notification” or “feedback” includes internal signal in a computing device.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 9 and 16-17 is/are rejected under 35 U.S.C. 102 (a)(1)/102 (a)(2) as being anticipated by Liao (US Patent 11,403,860 B1).
As per claim 9, Liao teaches a computing device (Abstract), comprising:
a sensor assembly (FIG. 1 LiDAR assembly 100, a camera assembly 200; col. 2 ln 10-18); and
a processor (FIG. 8) configured to:
capture, via the sensor assembly, a three-dimensional image depicting an object (Abstract “The system uses a LiDAR system and a monocular camera to obtain point cloud data and image data of an object, respectively”; col. 2 ln 46-51);
capture, via the sensor assembly, a two-dimensional image depicting the object (Abstract “The system uses a LiDAR system and a monocular camera to obtain point cloud data and image data of an object, respectively”; col. 2 ln 46-51);
determine a region of interest in the two-dimensional image, the region of interest containing the object (col. 4 ln 36-41);
determine, based on the region of interest from the two-dimensional image, a quality indicator corresponding to the three-dimensional image (Liao detects 2D bounding box in 2D image data and 3D bounding box in point cloud data (See above). Liao then transforms a point cloud data coordinate in the 3D object bounding box to a 2D pixel coordinate in the pixel coordinate system. Liao further calculates the distance between the centroid of the 3D bounding box that is transformed into 2D coordinate, and the center of the 2D bounding box. See col. 5 ln 40-col. 6 ln 45. The distance is considered the quality indicator.);
compare the quality indicator to a predetermined threshold; and
when the quality indicator does not satisfy the predetermined threshold, generate a feedback notification (col. 6 ln 53-col. 7 ln 3).
As per claim 16, dependent upon claim 9, Liao teaches the sensor assembly comprises a depth sensor configured to capture the three-dimensional image (FIG. 1 LiDAR assembly 100; col. 2 ln 46-51), and an image sensor configured to capture the two-dimensional image (FIG. 1 camera assembly 200; col. 2 ln 46-51).
As per claim 17, dependent upon claim 9, Liao teaches the sensor assembly includes a sensor configured to capture the three-dimensional image and the two-dimensional image (See rejections applied to claim 16).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 7-8, 10-11, 15 and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Liao, in view of KIMURA (US Publication 2025/0033899 A1).
As per claim 1, Liao teaches the invention substantially as claimed including a method (Abstract), comprising:
capturing a three-dimensional image depicting an object (See rejections applied to claim 9 above);
capturing a two-dimensional image depicting the object (See rejections applied to claim 9 above);
determining a region of interest in the two-dimensional image, the region of interest containing the object (See rejections applied to claim 9 above);
determining, based on the region of interest from the two-dimensional image, a quality indicator corresponding to the three-dimensional image (See rejections applied to claim 9 above);
comparing the quality indicator to a predetermined threshold object (See rejections applied to claim 9 above); and
when the quality indicator does not satisfy the predetermined threshold, generating a notification (See rejections applied to claim 9 above).
Liao, however, does not teach the notification is a positional notification.
KIMURA in an analogous field discloses a method for object loading recognition associated with a plurality of objects (Abstract). KIMURA’s system includes a sensor for measuring distances between the sensor and the plurality of objects, and a linear slider, which can be moved linearly by the linear slider (FIG. 8). KIMURA’s system is configured to: measure surfaces of the plurality of objects using the sensor; recognize dimensions, positions, and orientations of the plurality of objects based on the measured surfaces to identify recognized objects; calculate a confidence of each of the recognized objects; identify undistinguishable objects from the recognized objects based on the calculated confidences; calculate an approachable distance for each of the undistinguishable objects; and move the sensor towards the plurality of objects by a distance that corresponds to a minimum approachable distance from the calculated approachable distances (Abstract; FIG. 8-9). The calculated confidence of each of the recognized objects is considered a quality indicator. Upon determining the calculated confidence is below a confidence threshold, the object is considered as “undistinguishable”. The system then controls the vision sensor to move to a new position in order to better recognize the object. See FIG. 8-9, para. [0036]-[0040].
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to consider modify the teaching of Liao to incorporate the teaching of KIMURA to generate a positional notification. Doing so would set the image sensor in an appropriate position in order to increase the confidence of each recognized object as recognized by KIMURA (para. [0038]).
As per claim 2, dependent upon claim 1, Liao in view of KIMURA teaches determining the quality indicator comprises:
mapping the region of interest to a portion of the three-dimensional image (Liao col. 4 ln 22-36); and
determining a distance from a depth sensor to the object, based on the portion of the three-dimensional image (KIMURA FIG. 10; para. [0036]) .
As per claim 3, dependent upon claim 1, Liao in view of KIMURA teaches the predetermined threshold includes a lower distance threshold, and an upper distance threshold; and wherein the quality indicator satisfies the predetermined threshold when the distance is between the lower distance threshold and the upper distance threshold (KIMURA teaches the quality indicator is correlated with distance (para. [0038]). FIG. 10 shows the distance is between a minimum approachable distance and a maximum approachable distance; para. [0037]-[0040]).
As per claim 7, dependent upon claim 1, Liao in view of KIMURA teaches: when the quality indicator satisfies the predetermined threshold, determining dimensions of the object from the three-dimensional image (KIMURA FIG. 9 S1002 para. [0036]).
As per claim 8, dependent upon claim 1, Liao in view of KIMURA teaches capturing the three-dimensional image includes capturing a plurality of depth measurements, and generating a point cloud from the depth measurements (Liao FIG. 1; col. 2 ln 46-49 “In a second step at 600, according to some embodiments, the system 10 acquires point cloud data using the LiDAR assembly 100 and image data using the camera assembly 200 of one or more objects.”).
Claim 10, dependent upon claim 9, is rejected as applied to claim 2 above.
Claim 11, dependent upon claim 10, is rejected as applied to claim 3 above.
Claim 15, dependent upon claim 9, is rejected as applied to claim 7 above.
As per claim 18, an independent claim, Liao teaches a method (Abstract), comprising:
obtaining a point cloud depicting an object (See rejections applied to claim 1);
obtaining a two-dimensional image of the object (See rejections applied to claim 1);
detecting the object in the two-dimensional image (See rejections applied to claim 1;
based on the detected object in the two-dimensional image, determining a quality indicator corresponding to the three-dimensional image (See rejections applied to claim 1); and
generating a notification when the quality indicator does not satisfy a threshold (See rejections applied to claim 1).
Liao does not teach determining a quality indicator configured to indicate a likelihood that dimensions of the object can be obtained from the point cloud; and selecting, based on the quality indicator, between (i) generating feedback to reposition a sensor relative to the object and (ii) obtaining dimensions of the object from the point cloud.
KIMURA in an analogous field discloses a method for object loading recognition associated with a plurality of objects (Abstract). KIMURA’s system includes a sensor for measuring distances between the sensor and the plurality of objects, and a linear slider, which can be moved linearly by the linear slider (FIG. 8). KIMURA’s system is configured to: measure surfaces of the plurality of objects using the sensor; recognize dimensions, positions, and orientations of the plurality of objects based on the measured surfaces to identify recognized objects; calculate a confidence of each of the recognized objects; identify undistinguishable objects from the recognized objects based on the calculated confidences; calculate an approachable distance for each of the undistinguishable objects; and move the sensor towards the plurality of objects by a distance that corresponds to a minimum approachable distance from the calculated approachable distances (Abstract; FIG. 8-9). The calculated confidence of each of the recognized objects is considered a quality indicator (para. [0037]). The recognition includes recognition of the objects' sizes, positions, and orientations (FIG. 10 S1002). Upon determining the calculated confidence is below a confidence threshold, the object is considered as “undistinguishable” (i.e., the likelihood that dimensions of the object can be obtained is low). The system then controls the vision sensor to move to a new position in order to better recognize the object. See FIG. 8-9, para. [0036]-[0040].
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to consider modify the teaching of Liao to incorporate the teaching of KIMURA to determine a quality indicator to indicate a likelihood that dimensions of the object can be obtained from the measured data, and select, based on the quality indicator, generating feedback to reposition a sensor relative to the object. Doing so would set the image sensor in an appropriate position in order to increase the confidence of each recognized object as recognized by KIMURA (para. [0038]).
Claim(s) 4 and 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Liao, in view of KIMURA, as applied to claim 1 above, and further in view of Yu et al. (US Publication 2022/0371200 A1, hereafter Yu) and Sanchez (US Patent 10,643,441 B1).
As per claim 4, Liao in view of KIMURA teaches estimating dimension of the object; and selecting the predetermined threshold based on the estimated dimension (KIMURA FIG. 10; para. [0036]-[0040]).
Yu in an analogous field discloses a robotic system for object size measurement (Abstract). Specifically, Yu teaches determining, based on the two-dimensional image and the motion data, an estimated dimension of the object (para. [0128], [0132]).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to consider modify the teaching of Liao and KIMURA to incorporate the teaching of Yu to determine based on the two-dimensional image and the motion data, an estimated dimension of the object. The motivation of doing so is that object dimension is closely related to motion data, such as movement distance as recognized by Yu (para. [00121]).
Liao in view of KIMURA and Yu, does not further teach obtaining motion data via a motion sensor.
Sanchez Is evidenced that obtaining motion data via a motion sensor is
well-known and practiced (Abstract).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to consider modify the teaching of Liao, KIMURA and Yu to incorporate the teaching of Sanchez to obtain motion data via a motion sensor. The motivation of doing so is to track an object’s motion (Sanchez col. 1 ln 34-39).
Claim 12, dependent upon claim 10, is rejected as applied to claim 4 above.
Claim(s) 5-6 and 13-14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Liao, in view of KIMURA, as applied to claim 1 above, and further in view of Yao (US Publication 2020/0211542 A1).
As per claim 5, Liao in view of KIMURA does not teach determining a fraction of a field of view of an image sensor occupied by the region of interest.
Yao teaches calculating a fraction of a face occupies an image, i.e., a fraction of a field of view of the image sensor occupied by the region of interest (para. [0020] “In embodiments including a single camera, the depth calculation module 120 may approximate an objects distance to the camera by determining a portion of the image occupied by the object. For example, if a person's face occupies 80% of an image, then the depth calculation module 120 may approximate that the person is in close proximity to the camera”).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to consider modify the teaching of Liao and KIMURA to incorporate the teaching of Yao to determine a fraction of a field of view of the image sensor occupied by the region of interest. Doing so would allow the estimation of distance between image sensor and the object to be available as recognized by Yao (para. [0020]).
As per claim 6, dependent upon claim 5, Liao in view of KIMURA and Yao teaches the predetermined threshold includes a lower threshold, and an upper threshold (KIMURA teaches the quality indicator is correlated with distance (para. [0038]). FIG. 10 shows the distance is between a minimum approachable distance and a maximum approachable distance (para. [0037]-[0040]). KIMURA does not expressly teach a lower threshold and a upper threshold with respect to a fraction. However, Yao teaches that the fraction is related to the distance. Therefore, a person with ordinary skill in the art would have appreciated that fraction threshold can be derived from the distance threshold as an obvious variation.).
Claim 13, dependent upon claim 9, is rejected as applied to claim 5 above.
Claim 14, dependent upon claim 13, is rejected as applied to claim 6 above.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Contact
Any inquiry concerning this communication or earlier communications from the examiner should be directed to XUEMEI G CHEN whose telephone number is (571)270-3480. The examiner can normally be reached Monday-Friday 9am-6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John M Villecco can be reached on (571) 272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/XUEMEI G CHEN/Primary Examiner, Art Unit 2661
Prosecution Insights