DETAILED ACTION
Response to Amendment
Claims 1-20 were previously pending. Applicant’s amendment filed August 10, 2026, has been entered in full. Claims 1, 12 and 18 are amended. No claims are added or cancelled. Accordingly, claims 1-20 are now pending.
Response to Arguments
Applicant traverses the rejections of claims 1 and 3-7 under 35 U.S.C. 102 over the Kounosu reference (Remarks filed August 10, 2026, hereinafter Remarks: Pages 9-12).
Applicant first argues that Kounosu’s determination of hole locations is not a “measurement” (Remarks: Pages 10-11). Examiner respectfully disagrees for substantially the same reasons provided in the Final Rejection (Page 3) – i.e., determining the location of a hole is within the BRI of a measurement.
Applicant next asserts that “there is nothing in Kounosu that discloses or suggests a ‘Euclidean measurement’ for the location of the holes.” (Remarks: Page 11). The basis for Applicant’s assertion is respectfully unclear, as Applicant makes no attempt to define the scope of a “Euclidean measurement” or explain how the scope of a “Euclidean measurement” differs from the scope of a “measurement” in general.
The adjective “Euclidean” refers to something that is according to the principles of Euclid – see attached definition. A Euclidean space is a space where the axioms laid down by Euclid (that parallel lines do not intersect, that the sum of angles of a triangle is 180°, etc.) are valid. We humans inhabit a Euclidean space.
The hole location measurement of Kounosu is performed within a Euclidean space. For example, the workpiece in Kounosu is a real-world object with width, depth, and height (e.g., Fig. 4B), the hole locations are found within a two-dimensional image that is a Euclidean space (e.g., Fig. 4B), the pan and tilt angles used by Kounosu ([0032]) are angles within a Euclidean space, and so on. For at least these reasons, the hole location measurement of Kounosu falls within the BRI of a Euclidean measurement.
Applicant traverses the rejections of claims 12-20 under 35 U.S.C. 102 over the Shin reference (Remarks: Pages 12-14).
First, Applicant argues that Shin’s measurement of distance between points at intersections between an axis and a target object’s edges “is not a measurement of a ‘target object’ at least because the axes, even if intersecting the edge of an object, is measuring a distance between points of intersection of the edge of the object and the axes and not the edge of the object itself.” (Remarks: Page 12). Examiner respectfully disagrees. The intersecting points lie not only on the axes, but also the target object. Determining a distance between two points on an object is plainly a measurement of that object.
Next, Applicant argues that Shin’s measurement is not a Euclidean measurement (Remarks: Pages 12-14). In particular, Applicant argues that Shin does not measure, for example, the edge length of a bear (Remarks: Page 14). Examiner respectfully disagrees. The claims do not recite measuring an edge length. They simply require a “Euclidean measurement”. A measurement of a distance along a line between two points lying on an object is plainly a Euclidean measurement of that object, even if made within a two-dimensional plane. For example, a measurement of a box’s width as a distance between a point on a left side and a point on a right side is a Euclidean measurement of that box, even if the measurement does not consider, for example, the depth of the box.
Next, Applicant argues that Shin does not necessarily use a Euclidean measurement (Remarks: Page 14). Examiner respectfully disagrees. A distance between two points on a line is plainly within the scope of a Euclidean measurement.
Applicant traverses the previous rejections under 35 U.S.C. 103 for substantially the same reasons discussed above (Remarks: Page 15). Examiner respectfully disagrees for substantially the same reasons discussed above.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1 and 3-7 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by ‘Kounosu’ (WO 2021/177245 A1; cited in parent application no. 17/411,339).
Regarding claim 1, Kounosu discloses a dimensional measurement method based on deep learning (e.g., Figure 7; see further mapping below), the method comprising:
capturing, by a processor (e.g., [0013], Fig. 1, image processing device 10, which includes CPU), an image of a target object according to a preset location precision to obtain an image (e.g., [0015], [0017], [0035], Fig. 7, step S1, wide angle image of work target is captured), wherein the location precision indicates an imaging resolution of one or more location points used to measure a specified portion of the target object (e.g., [0019], the location points used to measure the target object are positions of holes; [0017]-[0018], Fig. 3A, the preset wide-angle configuration indicates a relatively lower imaging resolution of the holes, when compared to the narrow-angle configuration shown in Fig. 3B; I.e., each hole is covered by less image area in the wide angle view as opposed to the narrow-angle view);
determining, by the processor (see above), at least one target region from the image (e.g., [0018], Fig. 3A, area 201 is determined as a target region), with each target region including at least one of the one or more location points (Fig. 3A, the target region 201 includes the holes);
processing, by the processor (see above), at least one target region using a pre-trained neural network to obtain first position information of each location point (e.g., [0019], Fig. 3B, machine learning is used to obtain positions of each hole in image 102 of the target region; Fig. 6, [0021] et seq., the machine learning is a pre-trained neural network); and
determining, by the processor (see above), a Euclidean measurement of the specified portion of the target object ([0032], position of each hole with respect to workpiece is determined; The hole location measurement is performed within a Euclidean space; For example, the workpiece in Kounosu is a real-world object with width, depth, and height (e.g., Fig. 4B), the hole locations are found within a two-dimensional image that is a Euclidean space (e.g., Fig. 4B), the pan and tilt angles ([0032]) are angles within a Euclidean space, and so on; For at least these reasons, the hole location measurement falls within the BRI of a Euclidean measurement) according to the location precision ([0032], hole positions are determined based on pan, tilt, and zoom of camera, which is the mechanism for capturing wide or narrow angle images – [0017], [0019]) and the first position information of each location point ([0032], hole positions are determined based on their positions in the image detected by the machine learning).
Regarding claim 3, Kounosu discloses the method according to claim 1, wherein when the range of positions of the location points is known (e.g., Figs. 3A-B, the holes are known to be within the bounds of the workpiece W), and the number of location points used in measuring the target object is one (While the specific example shown in Figs. 3A-B have multiple holes, the target region is defined the same way [i.e., as the region of the whole workpiece] regardless of the number of holes, including if there was one hole), the processor identifies a region corresponding to a range of positions of the location point in the image as the target region ([0018], Fig. 3A, range 201), and wherein the range of positions of the location point indicates a smallest area where the location points appear in a field of view of the image ([0018], Fig. 3A, target range 201 is bounding box – i.e. smallest rectangular area that includes workpiece where holes appear in field of view of image), and the field of view indicates the target region captured when the image capture component captures an image of the target object (e.g., Fig. 3A shows target region within field of view; e.g., Fig. 3B also shows a narrower field of view of an image captured to show only the target region).
Regarding claim 4, Kounosu discloses the method according to claim 1, wherein determining, by the processor, the at least one target region from the said image to be processed, further comprises:
when a range of positions of the location points is known (e.g., Figs. 3A-B, the holes are known to be within the bounds of the workpiece W, and within image 101), and a number of location points used in measuring the target object is greater than one (e.g., Figs. 3A-B show examples where the number of holes – i.e., the number of location points – is greater than one):
determining, by the processor, multiple selections that choose target regions (e.g., [0017], Fig. 3A, image 101 is taken to capture workpiece target region; The region covered by image 101 is a first selection of a target region; e.g., [0018], Fig. 3A, machine learning is applied to select target region 201 within the image; The bounding box region 201 is a second, more-refined selection of a target region), based on the range of positions of each location point (both selections are determined to find the workpiece, which is the range of positions in which each location point occurs).
Regarding claim 5, Kounosu discloses the method according to claim 4, wherein determining, by the processor, the at least one target region from the said image to be processed, further comprises:
when a range of positions of the location points is known (e.g., Figs. 3A-B, the holes are known to be within the bounds of the workpiece W, and within image 101), and a number of location points used in measuring the target object is greater than one (e.g., Figs. 3A-B show examples where the number of holes – i.e., the number of location points – is greater than one):
determining, by the processor, a total area of the target regions under the respective selections (e.g., Fig. 3A, both image 101 and bounding box 201 define total areas of the target regions for each respective selection).
Regarding claim 6, Kounosu discloses the method according to claim 4, wherein determining, by the processor, the at least one target region from the said image to be processed, further comprises:
when a range of positions of the location points is known (e.g., Figs. 3A-B, the holes are known to be within the bounds of the workpiece W, and within image 101), and a number of location points used in measuring the target object is greater than one (e.g., Figs. 3A-B show examples where the number of holes – i.e., the number of location points – is greater than one):
setting, by the processor, a selection that has a smallest total area as the target selection (e.g., [0018], Fig. 3A, bounding box region 201 is selected and this has a smaller total area than the image region 101 within which it is placed).
Regarding claim 7, Kounosu discloses the method according to claim 4, wherein determining, by the processor, the at least one target region from the said image to be processed, further comprises:
when a range of positions of the location points is known (e.g., Figs. 3A-B, the holes are known to be within the bounds of the workpiece W, and within image 101), and a number of location points used in measuring the target object is greater than one (e.g., Figs. 3A-B show examples where the number of holes – i.e., the number of location points – is greater than one):
determining, by the processor, at least one target region from the image by a target selection (e.g., [0018], Fig. 3A, target region 201).
Claim(s) 12-20 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by ‘Shin’ (“Integration of deep learning-based object recognition and robot manipulator for grasping objects,” 2019).
Regarding claim 12, Shin discloses a dimensional measurement method based on deep learning (e.g., Fig. 2), comprising:
capturing, by a camera (e.g., Fig. 2, RGB-D Camera; Section V, 1st paragraph), an image of a target object (e.g., Figs. 3 and 4, image of various target objects);
identifying, by a processor (see, e.g., Fig. 2, PC; Sec. II.B, last par.; Sec. V, 1st par.), one or more target regions within the image of the target object (e.g., Fig. 2, Mask R-CNN; Sec. III, Mask R-CNN is applied on each frame, which provides masks identifying target regions within the image; Fig. 4(c) shows an example of the mask output [best seen in color]) that include one or more location points (e.g., Sec. III, 2nd-to-last par., points along the edges of the regions are location points used for measuring);
identifying, by the processor using deep learning, the one or more location points within the one or more target regions (e.g., Sec. III, 2nd-to-last par., results of Mask R-CNN deep learning are used to set a center of gravity and straight lines are drawn within the region to find points that meet the edge of the object; The points that meet the edge of the object are the location points, and the line-drawing process is the identifying of the location points);
determining, by the processor, a Euclidean measurement of at least a portion of the target object identified by the one or more target regions within the image of the target object (e.g., Sec. III, 2nd-to-last par., “the axis having the shortest distance between the two points of the six axes is selected for grasping orientation”; The determination of the distances between the points along the axis is a measurement of at least a portion of the target object at least because, as noted above, the points lie on the object’s edges; I.e., the distances between the points are measurements of the length of the target object along each of the axes; The measurement is “Euclidean” at least because it follows standard Euclidean geometry – e.g., the points lie on a line in a two-dimensional space).
Regarding claim 13, Shin discloses the dimensional measurement method based on deep learning of claim 12, further comprising:
identifying, by the processor, a second set of one or more location points in the one or more target regions with in the image of the target object (e.g., Sec. III, 2nd-to-last par., six axes are drawn, and two edge points are found for each axis; The edge points are location points, and the edge points found for each axis are a different set; I.e., the six axes produce six sets of edge/location points, one of which is a “second” set); and
determining, by the processor, a second dimensional measurement of the target object based on the target object (e.g., Sec. III, 2nd-to-last par., dimensional measurements of the target object are determined for each of the six axes/sets of location points in order to select a grasping orientation).
Regarding claim 14, Shin discloses the dimensional measurement method based on deep learning method of claim 12, further comprising:
adjusting an orientation of the target object, by robotic manipulation, based on the determined dimensional measurement (e.g., Sec. III, 2nd-to-last par., dimensional measurement is used to determine grasping orientation; e.g., Figs. 6-7, Sec. V, target object such as the teddy bear is picked up and placed in bin by robotic grasping manipulation, its orientation being adjusted during this process).
Regarding claim 15, Shin discloses the dimensional measurement method based on deep learning method of claim 12, further comprising:
placing the target object, by robotic manipulation, in another part based on the determined dimensional measurement (e.g., Sec. III, 2nd-to-last par., dimensional measurement is used to determine grasping orientation; e.g., Figs. 6-7, Sec. V, target object such as the teddy bear is picked up and placed in bin by robotic grasping manipulation, the bin being the “another part”).
Regarding claim 16, Shin discloses the dimensional measurement method based on deep learning method of claim 12, further comprising:
providing, by the processor, dimensional data of the target object (e.g., Sec. III, 2nd-to-last par., dimensional data is provided for selection of grasping orientation).
Regarding claim 17, Shin discloses the dimensional measurement method based on deep learning method of claim 12, wherein the identifying the one or more location points within the one or more target regions further includes identifying first position information of each of the one or more location points by deep learning in a neural network (e.g., Fig. 2, Mask R-CNN is a deep learning neural network; e.g., Sec. III, 2nd-to-last par., Mask R-CNN deep learning is used to define the positions of the center of gravity and axes on which the object edge location points are identified; For at least this reason, first position information of each of the points is identified by deep learning in a neural network).
Regarding claim 18, Examiner notes that the claim recites a device, comprising a camera which captures an image of a target object; a memory device; and a processor which: implements a method that is substantially the same as the method of claim 12.
Shin discloses the method of claim 12 (see above).
Shin further discloses implementing its method as a device, comprising a camera which captures an image of a target object (e.g., Fig. 2, RGB-D Camera; Section V, 1st paragraph); a memory device (e.g., Sec. V, 1st par., main PC and/or GTX 1080Ti GPUs); and a processor (e.g., Sec. V, 1st par., main PC and/or GTX 1080Ti GPUs).
Accordingly, Shin also discloses the invention of claim 18 for substantially the same reasons as claim 12.
Regarding claim 19, Examiner notes that the claim recites limitations that are substantially the same as limitations recited in claim 13. Shin discloses the invention of claim 13. Accordingly, claim 19 is also rejected under 35 U.S.C. 102(a)(1) as being anticipated by Shin for substantially the same reasons as claim 13.
Regarding claim 20, Examiner notes that the claim recites limitations that are substantially the same as limitations recited in claim 17. Shin discloses the invention of claim 17. Accordingly, claim 20 is also rejected under 35 U.S.C. 102(a)(1) as being anticipated by Shin for substantially the same reasons as claim 17.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 2 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kounosu in view of ‘Carion’ (“End-to-End Object Detection with Transformers,” 2020; cited in parent application no. 17/411,339).
Regarding claim 2, Kounosu teaches the method of claim 1 (see above).
Kounosu obtains position information of location points using an object detection neural network (e.g., [0019], [0021]). Kounosu teaches several examples of object detection neural networks ([0027]) and teaches that various object detections based on machine learning can be used, without being limited to the given examples ([0027]). Nevertheless, Kounosu does not explicitly teach an example where the neural network includes at least one sub-network that corresponds to the target region, and the said sub-network includes an encoder and a decoder, and the pre-trained neural network processes at least one target region to obtain position information of each location point, and wherein the processor performs feature extraction on the at least one target region by obtaining a feature map of the target region by the encoder in the sub-network corresponding to the target region; and wherein the processor processes the feature map to obtain position information of each location point in the target region by the decoder in the sub-network.
However, Carion does teach an approach for object detection based on machine learning (e.g., Fig. 2, DETR), where a neural network (Fig. 2, DETR network) includes at least one sub-network that corresponds to the target region (Fig. 2, encoder-decoder subnetwork that processes input image, which is the target region in Kounosu), and the said sub-network includes an encoder and a decoder (Fig. 2, encoder and decoder), and the pre-trained neural network processes at least one target region to obtain position information of each location point (Fig. 2, right, object detections), and wherein the processor performs feature extraction on the at least one target region by obtaining a feature map of the target region by the encoder in the sub-network corresponding to the target region (e.g., Fig. 2, image is processed through backbone and then through encoder, which obtains transformed feature map of input image – i.e. the target region); and wherein the processor processes the feature map to obtain position information of each location point in the target region by the decoder in the sub-network (Fig. 2, decoder processes features passed from encoder in order to predict object positions marked by bounding boxes).
Carion teaches that its DETR model achieves similar performance to Faster RCNN (Section 4.1), which is one of the types of models suggested by Kounosu ([0027]), and provides an advantageously simplified detection (e.g., Page 214, middle paragraph).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to modify the method of Kounosu with the DETR object detection of Carion in order to improve the method with the reasonable expectation that this would result in a method that used a suitable object detector with an advantageously simplified detection pipeline. This technique for improving the method of Kounosu was within the ordinary ability of one of ordinary skill in the art based on the teachings of Kounosu and Carion.
Therefore, it would have been obvious to one of ordinary skill in the art to combine the teachings of Kounosu and Carion to obtain the invention as specified in claim 2.
Claim(s) 8-11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kounosu in view of ‘Růžička’ (“Fast and accurate object detection in high resolution 4K and 8K video using GPUs,” 2018; cited in parent application no. 17/411,339).
Regarding claim 8, Kounosu teaches the method of claim 1 (see above).
Kounosu teaches, when the range of positions of the location points is unknown, applying a machine learning object detector to determine at least one target region (e.g., [0018], Fig. 3A). As the object detector, Kounosu lists examples including RCNN, YOLO and SSD ([0027]), but states that other object detectors can be used ([0028]).
Kounosu does not explicitly teach
downsampling, by the processor, the image according to a preset downsampling ratio to obtain an intermediate image.
However, Růžička does teach an object detector that identifies a target region of an image by:
downsampling, by the processor, the image according to a preset downsampling ratio to obtain an intermediate image (e.g., Fig. 3, stage I, downsampling from 2160x2160 to 608x608).
Růžička teaches that current state of the art models, such as YOLO, are “focused on working with low-resolution images” (Sec. 1, 2nd and 3rd pars.), but that such low-resolution images lose a lot of detail relative to what can be captured with modern high-resolution cameras (Sec. 1, 4th par.). Růžička teaches that its approach can achieve fast performance without losing details such as small objects due to downscaling (Sec. 4).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to modify the method of Kounosu with the object detector of Růžička in order to improve the method with the reasonable expectation that this would result in a method whose object detection was fast and advantageously able to detect small details. This technique for improving the method of Růžička was within the ordinary ability of one of ordinary skill in the art based on the teachings of Kounosu and Růžička.
Therefore, it would have been obvious to one of ordinary skill in the art to combine the teachings of Kounosu and Růžička to obtain the invention as specified in claim 8.
Regarding claim 9, Kounosu in view of Růžička teaches the method of claim 8, and Růžička further teaches when determining by the processor at least one target region from the image to be processed (i.e., as part of object detection – see rejection of claim 8), further comprising:
determining, by the processor, second position information of each location point in the intermediate image (e.g., Fig. 3, stage I, initial YOLO object detection produces detections in downscaled image; In the context of Kounosu, these detections are positions of workpiece, which are coarse positions of each location point since they are all included in the workpiece).
Regarding claim 10, Kounosu in view of Růžička teaches the method of claim 9, and Růžička further teaches when determining by the processor at least one target region from the image to be processed (i.e., as part of object detection – see rejection of claim 8), further comprising:
determining, by the processor, third position information of each location point in the image to be processed according to the second position information, wherein the third position information indicates the positions of the location points in the image to be processed, the position information of the location points in the image corresponding to the second position information (e.g., Fig. 3, stage II, bounding boxes from stage I YOLO are placed on corresponding positions of full-resolution input image).
Regarding claim 11, Kounosu in view of Růžička teaches the method of claim 10, and Růžička further teaches when determining by the processor at least one target region from the image to be processed (i.e., as part of object detection – see rejection of claim 8), further comprising:
determining, by the processor, at least one target region from the image to be processed, according to the third position information (e.g., Fig. 3, processing continues through remainder of stage II and stage III to produce output bounding boxes) and preset dimensions (e.g., Fig. 3, the processing to produce the output target region bounding boxes is based on various preset dimensions, such as the preset model input dimensions of 608x608 pixels).
Conclusion
The following prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 2023/0360193 A1
United States national stage entry of the ‘Kounosu’ PCT publication reference
Provides an English translation
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GEOFFREY E SUMMERS whose telephone number is (571)272-9915. The examiner can normally be reached Monday-Friday, 7:00 AM to 3:30 PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached at (571) 272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GEOFFREY E SUMMERS/Examiner, Art Unit 2669
Prosecution Insights