DETAILED ACTION
This Office Action is taken in response to Applicant’s Amendment and Remarks filed on
10/09/2025 regarding Application No. 17/901,739 originally filed on 09/01/2022. Claims 1-20 are pending for consideration:
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Regarding the rejection of claims 1–20 under 35 U.S.C. 101 as being directed to an abstract idea without significantly more, the applicant argues “claim 1 does not recite a Mental Process… because the claimed features cannot be practically performed in the human mind,” and further argues the claims involve “machine or computer readable information… used to operate a robotic system,” and are analogous to Research Corp. [Remarks, p. 10–11]. The examiner respectfully disagrees.
The claims recite, at a high level, collecting data, generating results, determining classifications (occlusion/contested region), generating mask information, and deriving a motion plan, which under broadest reasonable interpretation encompasses evaluation and selection steps that can be performed as mental judgments (i.e., classifying regions, excluding regions from a mask, selecting a target and a grip location, and specifying a trajectory as an output plan). The recitation of “image data,” “detection mask information,” and “motion plan” does not, by itself, impose a meaningful technological limitation; these are results and information outputs rather than a recited improvement to a computer or robot architecture. Further, the claims do not recite any particular computer-implemented technique (e.g., a specific algorithm, data structure, sensor control, or robot-control improvement) that would integrate the judicial exception into a practical application under MPEP 2106.04/2106.05. Accordingly, the §101 rejection is maintained.
Regarding the rejection of claims 1–20 under 35 U.S.C. 103 over Humayun in view of Ku and Xu, the applicant argues the combination “fails to disclose or suggest… ‘wherein the contested region has (1) dimensions less than a smallest expected dimension, (2) a non-rectangular shape, or both,’” and further argues Xu is “silent” as to such features and is “categorically unrelated” because it is directed to video surveillance [Remarks, p. 12]. The examiner respectfully disagrees.
Xu expressly discloses identifying an overlapping zone (i.e., disputed pixels present in more than one probability mask) and using that zone in resolving ownership of pixels between merged objects (as per “determine ‘disputed pixels’… having non-zero value in more than one… probability masks… called the ‘overlapping zone’” in ¶95; and as per classifying pixels to the object returning the highest likelihood in ¶83). Xu’s overlapping/“disputed pixels” zone is an occlusion/contested region as claimed, and its pixel-region form inherently yields a non-rectangular region as depicted/derived from object masks (see ¶95 and Fig. 9 as applied in the rejection). Further, the claimed “dimensions less than a smallest expected dimension” limitation does not clearly provide an objective structural distinction over Xu’s overlapping zone because “smallest expected dimension” is itself indefinite (see §112(b) rejection of claims 1, 8, and 15); as such, applicant’s traversal based on that phrase is not persuasive to establish patentable distinction. Applicant’s arguments regarding field-of-use (surveillance vs. robotics) are also unavailing because the applied teachings concern occlusion/overlap segmentation and assignment of disputed regions between objects, which are techniques applicable to object-instance separation in image-based processing regardless of the end application. Accordingly, the §103 rejection is maintained.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim(s) 1, 8, and 15 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The claim(s) recite: “wherein the contested region has (1) dimensions less than a smallest expected dimension ….” The phrase “smallest expected dimension” lacks a clear, objective boundary. The claim does not specify what “expected” is based on, which object(s) the “smallest” dimension is drawn from, or the measurement domain/reference frame for the “dimensions” (e.g., image space vs. world/depth space). As a result, the scope of the limitation is unclear and the metes and bounds of the claim cannot be determined with reasonable certainty. Therefore, claim(s) 1, 8, and 15 are indefinite.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
1. A method of operating a robotic system, the method comprising:
generating detection features based on image data representative of one or more flexible objects at a start location;
generating a detection result corresponding to the one or more flexible objects based on the detection features;
wherein the detection result is generated at least partially based on the one or more flexible objects having a polygonal shape,
wherein the detection result indicates presence and location of positively identified objects;
based on the image data and the detection result, determining whether the detection result indicates an occlusion region, including whether the occlusion region is a contested region, wherein the occlusion region represents an overlap between an instance of the one or more flexible objects and a further instance of the one or more flexible objects,
wherein the contested region has (1) dimensions less than a smallest expected dimension, (2) a non-rectangular shape, or both;
determining, when the occlusion region is the contested region, whether the contested region is a part of the instance of the one or more flexible objects or the further instance of the one or more flexible objects based on surface continuity, surface features, or a combination thereof;
generating detection mask information for the detection result, wherein the detection mask information includes positive identification information that does not include the occlusion region; and
wherein the detection mask information represents a region where one object is positively depicted without occlusion and is detected;
deriving a motion plan for the target object, wherein the motion plan includes:
a target object selected from the one or more flexible objects based on the detection mask information
a grip location, on the target object, for an end-effector of a robotic arm based on the detection mask information, and
one or more trajectories for the robotic arm for transferring the target object from the start location to a destination location.
101 Analysis - Step 1: Statutory category – Yes
The claims recites a method including at least one step. The claims falls within one of the four statutory categories. MPEP 2106.03
Step 2A Prong one evaluation: Judicial Exception – Yes – Mental processes
Claim(s) is to be analyzed to determine whether it recites subject matter that falls within one of the following groups of abstract ideas: a) mathematical concepts, b) mental processes, and/or c) certain methods of organizing human activity.
The Office submits that the foregoing bolded limitation(s) constitutes judicial exceptions in terms of “mental processes” because under its broadest reasonable interpretation, the claim covers performance using mental processes.
The Office submits that the foregoing bolded limitation(s) constitutes judicial exceptions in terms of “mental processes” because under its broadest reasonable interpretation, the claim covers performance using mental processes.
The claims recite the limitations “generating…”, “determining…”, and “deriving…”. The “generating…”, “determining…”, and “deriving…” limitations, as drafted, are processes that, under their broadest reasonable interpretation, cover performance of the limitation in the mind but for the recitation of “image data” and “target object, grip location, one or more trajectories”, nothing in the claims precludes the step from practically being performed in the mind. For example, but for the “image data” and “target object, grip location, one or more trajectories” language, the claim encompasses looking at data collected and forming a simple judgement. The mere nominal recitation of image data and target object, grip location, and generated trajectories do not take the claim limitations out of the mental process grouping. Thus, the claims recite a mental process.
101 Analysis - Step 2A Prong two evaluation: Practical Application - No
Claim(s) is evaluated whether as a whole it integrates the recited judicial exception into a practical application. As noted in the 2019 PEG, it must be determined whether any additional elements in the claims beyond the abstract idea integrate the exception into a practical application in a manner that imposes a meaningful limit on the judicial exception. The courts have indicated that additional elements merely using a computer to implement an abstract idea, adding insignificant extra solution activity, or generally linking use of a judicial exception to a particular technological environment or field of use do not integrate a judicial exception into a “practical application.”
In the present case, the additional limitations beyond the above-noted abstract idea are as follows (where the underlined portions are the “additional limitations” while the bolded portions continue to represent the “abstract idea”)
The claims recite the additional elements of “image data” and “target object, grip location, one or more trajectories” that performs the “generating…”, “determining…”, and “deriving…” steps. The steps by the additional elements are recited at a high level of generality and merely automates the steps, therefore acting as a generic computer to perform the abstract idea. The additional elements are claimed generically and is operating in its ordinary capacity and does not use the judicial exception in a manner that imposes a meaningful limit on the judicial exception, such that the claims are more than a drafting effort designed to monopolize the exception. The additional limitations are no more than mere instructions to apply the exception using a computer.
Accordingly, even in combination, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
The claims are directed to the abstract idea.
Step 2B evaluation: Inventive Concept: - No
The claim(s) are evaluated whether the claim as a whole amount to significantly more than the recited exception, i.e., whether any additional element, or combination of additional elements, adds an inventive concept to the claims.
As discussed with respect to Step 2A Prong Two, the additional elements in the claims amount to no more than mere instructions to apply the exception using a generic computer component. The same analysis applies here in 2B, i.e., mere instructions to apply an exception on a generic computer cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B, MPEP 2106.05(f).
Therefore, Claim 1 is ineligible.
Dependent claim(s) 2-7 do not recite any further limitations that cause the claim(s) to be patent eligible. Rather, the limitations of dependent claims are directed toward additional aspects of the judicial exception and/or generic additional elements that do not integrate the judicial exception into a practical application. Claims 2-7 recite limitations that are insignificant extra-solution activity as they are nominally or tangentially related to the invention and well-known. Therefore, dependent claim(s) 2-7 are not patent eligible under the same rationale as provided for in the rejection of claim 1.
Independent claims 8 and 15 are rejected using the same rationale, mutatis mutandis, applied to Claim 1 above, respectively.
Dependent claims 9-14 and 16-20 are rejected using the same rationale, mutatis mutandis, applied to dependent claims 2-7 above, respectively.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Humayun (US Pub. No. 20220016766) in view of Ku (US Pub. No. 20220016767) in further view of Xu (US Pub. No. 20080166045).
As per Claim 1, Humayun discloses a method of operating a robotic system (as per “ the method and system can be used to select more accurate and successful object grasps (e.g., for packing robots, manufacturing robots, etc.)” in ¶20), the method comprising:
based on the image data and the detection result, determining whether the detection result indicates an occlusion region, wherein the occlusion region represents an overlap between an instance of the one or more flexible objects and a further instance of the one or more flexible objects (as per “preferably a ‘dense’ object scene, which can include a plurality of overlapping objects (e.g., where one or more objects are occluded by another object within the scene; the object scene can include a first plurality of objects that partially occludes a second plurality of objects; etc.)" in ¶49);
deriving a motion plan for the target object (as per "computing system can include a motion planner 148" in ¶43), wherein the motion plan includes:
a target object selected from the one or more flexible objects based on the detection mask information; a grip location, on the target object, for an end-effector of a robotic arm based on the detection mask information (as per "determining a set of candidate grasp points based on the object parameters; selecting a grasp point from the set of candidate grasp points;" in Claim 1, as per "end effector preferably functions to grip an object” in ¶29),
one or more trajectories for the robotic arm for transferring the target object from the start location to a destination location (as per “system can alternatively be otherwise suitably controlled and/or otherwise suitably enable end effector articulation. The robotic arm can include any suitable number of joints which enable articulation of the end effector in a single degree of freedom (DOF). The arm preferably includes 6 joints (e.g., a 6-axis robotic arm), but can additionally or alternatively include seven joints, more than seven joints, and/or any other suitable number of joints." in ¶30, as per "The control instructions can be determined by a grasp planner, which can determine a robotic end effector path, robotic end effector pose, joint waypoints (e.g., in cartesian/sensor coordinate frame, in a joint coordinate frame, etc.), and/or any other suitable control instructions." in ¶58).
Humayun does not expressly disclose or teach wherein:
generating detection features based on image data representative of one or more flexible objects at a start location
generating a detection result corresponding to the one or more flexible objects based on the detection features,
wherein the detection result is generated at least partially based on the one or more flexible objects having a polygonal shape,
determining including whether the occlusion region is a contested region,
wherein the contested region has (1) dimensions less than a smallest expected dimension, (2) a non-rectangular shape, or both;
determining, when the occlusion region is the contested region, whether the contested region is a part of the instance of the one or more flexible objects or the further instance of the one or more flexible objects based on surface continuity, surface features, or a combination thereof:
generating detection mask information for the detection result, wherein the detection mask information includes positive identification information that does not include the occlusion region;
wherein the detection mask information represents a region where one object is positively depicted without occlusion and is detected;
Ku discloses of a robotic system (as per “articulate the robot arm to grasp an object” in ¶29 ) wherein:
generating detection features based on image data representative of one or more flexible objects at a start location (as per “receiving an image of a scene comprising a plurality of visible objects” in Claim 1, as per “The output of the detector can include one or more features, keypoints (e.g., surface keypoints, unique object features, object bounding box features, etc.), labels (e.g., face labels, silhouette labels, texture labels, haptic labels), one or more object masks, one or more scores (e.g., a visibility score for each object, for each feature, etc.), and/or any other suitable information.” in ¶44, as per “The objects can be: rigid, deformable, matte, transparent, reflective, and/or have any other suitable property." in ¶37);
generating a detection result corresponding to the one or more flexible objects based on the detection features (as per “The one or more detected objects can represent object detections in a scene. ” in ¶63, as per “The objects can be: rigid, deformable, matte, transparent, reflective, and/or have any other suitable property." in ¶37).
wherein the detection result is generated at least partially based on the one or more flexible objects having a polygonal shape, (as per “The subregion can be rectangular, circular, annular, polygonal (e.g., convex polygon, irregular polygon, etc.), or any other suitable geometry” in ¶70)
wherein the detection result indicates presence and location of positively identified objects; (as per “The one or more detected objects can represent object detections in a scene. The detected objects can be represented by an object mask, an object bounding box, principal axis/axes (e.g., 2D projection into image coordinate frame, 3D, 6D, etc.), and/or any other suitable information” in ¶63, as per “using an object detector, generating a set of keypoints for an exposed portion of a visible object in the scene, wherein each keypoint defines a position within a coordinate frame of the image;” in Claim 1)
generating detection mask information for the detection result, wherein the detection mask information includes positive identification information that does not include the occlusion region; (as per “this can include assigning a low score (e.g., “0”) to keypoints of the object that overlap with the mask” in ¶59, as per “The subdivided keypoints are preferably the unoccluded keypoints (e.g., wherein the occluded keypoints, determined based on the respective occlusion scores, can be discarded, downweighed, or otherwise managed) and/or unoccluded portions of the object face, but can additionally or alternatively include the occluded keypoints.” in ¶76, as per “the object detector can be trained to only detect unoccluded keypoints depicted in the image.” in ¶61)
wherein the detection mask information represents a region where one object is positively depicted without occlusion and is detected; (as per “wherein determining the occlusion score can include: for each object, determining a mask from the upper object instances (e.g., detected within and/or depicted within the image; objects above the object of interest, as determined from visual cues or depth information; using the upper objects' detected boundaries; etc.); and determining an occlusion score based on the mask. In a first embodiment where the occlusion score is for the keypoint, this can include assigning a low score (e.g., “0”) to keypoints of the object that overlap with the mask, and assigning a high score (e.g., “1”) to keypoints of the object that do not overlap with the mask” in ¶59, as per “However, the occlusion score can be otherwise determined. The mask can optionally be used to verify that the grasp location (determined in S400) is accessible, such as by verifying that the grasp location does not overlap the mask” in ¶59)
In this way, the system operates to grasp objects in occluded environments (¶22). Like Humayun, Ku is concerned with robotic systems.
Therefore, from these teachings of Humayun and Ku, one of ordinary skill in the art before the effective filing date would have found it obvious to apply the teachings of Ku to the system of Humayun since doing so would generate detection features and results based on image data. (¶22) Such modification would also allow the system to generate an occlusion score for an object as well as selecting a most visible object face per detected object based on the occlusion scores. (¶17)
Humayun and Ku fail to expressly disclose:
determining including whether the occlusion region is a contested region,
wherein the contested region has (1) dimensions less than a smallest expected dimension, (2) a non-rectangular shape, or both;
determining, when the occlusion region is the contested region, whether the contested region is a part of the instance of the one or more flexible objects or the further instance of the one or more flexible objects based on surface continuity, surface features, or a combination thereof:
Xu discloses of tracking objects in a video sequence, comprising:
determining including whether the occlusion region is a contested region, (as per “When a merge between two or more objects is detected, a first-order model can be used to predict the centroid location of each object. The textural appearance of each object is correlated with the merged image at the centroid location to find a best fit. Given a best-fit location, a shape probability mask can then be used to determine ‘disputed pixels’, namely those pixels having non-zero value in more than one of the objects' probability masks. This group of pixels is called the ‘overlapping zone’” in ¶95)
wherein the contested region has (1) dimensions less than a smallest expected dimension, (2) a non-rectangular shape, or both; (as per “Given a best-fit location, a shape probability mask can then be used to determine ‘disputed pixels’, namely those pixels having non-zero value in more than one of the objects' probability masks. This group of pixels is called the ‘overlapping zone’. An illustration of the overlapping zone is shown schematically in FIG. 9. Once the overlapping zone is determined, objects are ordered so that those assigned fewer ‘disputed’ pixels are given greater depth” in ¶95, as per Fig. 9)
determining, when the occlusion region is the contested region, whether the contested region is a part of the instance of the one or more flexible objects or the further instance of the one or more flexible objects (as per “To summarise the method, for each pixel of the group blob, we calculate the likelihood of the pixel belonging to an individual blob forming part of the group blob. The likelihood calculation is based on the appearance model generated for that individual blob in attention level 2. This process is repeated for each of the blobs forming part of the group blob. Following this, the pixel is classified to the individual blob returning the highest likelihood value. The aim of the group segmentation stage 69 is illustrated in FIGS. 7(a) to 7(c) which show, respectively, (a) an original video frame, (b) the resulting group blob and (c) the ideal segmentation result. Having segmented the group blob, it is possible to maintain the identities of the two constituent objects during the occlusion Such that, when they split, no extra processing is required to re-learn the identities of the two objects” in ¶83) based on surface continuity (as per
PNG
media_image1.png
328
278
media_image1.png
Greyscale
¶86), surface features (as per ¶85), or a combination thereof: (as per ¶85-¶86, eq n(s) 4-7)
PNG
media_image2.png
572
290
media_image2.png
Greyscale
In this way, Xu operates to improve the group segmentation to ensure robust object tracking (¶97). Like Humayun and Ku, Xu is concerned with monitoring objects (as per Fig. 2 & Figs. 7A-7C).
Therefore, from these teachings of Humayun, Ku, and Xu, one of ordinary skill in the art before the effective filing date would have found it obvious to apply the teachings of Ku and Humayun to the system of Xu detect a contested region to determine which object instance the contested region belongs to (as per ¶83). Such modification would also allow the system to compare the status of blobs in the current frame and the respective status of objects already being tracked. (¶50)
As per Claim 2, the combination of Humayun, Ku, and Xu teaches or suggests all limitations of Claim 1. Humayun further discloses determining the grip location within an area on a surface of the target object corresponding to the positive identification information (as per "The output of the detector can include one or more features, keypoints (e.g., surface keypoints, unique object features, object bounding box features, etc.), labels (e.g., face labels, silhouette labels, texture labels, haptic labels), one or more object masks, one or more scores (e.g., a visibility score for each object, for each feature, etc.), and/or any other suitable information." in ¶36, as per " The grasp points can be selected from: a set of candidate grasp points 106 (e.g., selected by an upstream candidate selection module); from the graspability map (e.g., based on values associated with the image features); and/or otherwise selected. (e.g., as generated by the graspability network)." in ¶42).
As per Claim 3, the combination of Humayun, Ku, and Xu teaches or suggests all limitations of Claim 1. Humayun further discloses determining the grip location to avoid an area on a surface of the target object corresponding to occlusion information when the detection result includes the occlusion region (as per "In some variants, the grasp point can be selected by a failure hysteresis and/or loss function to avoid selecting points (e.g., by selective weighting, blacklisting, etc.) based on the proximity to a recent grasp failure." in ¶81, as per "The grasp points can be selected from: a set of candidate grasp points 106 (e.g., selected by an upstream candidate selection module); from the graspability map (e.g., based on values associated with the image features); and/or otherwise selected. (e.g., as generated by the graspability network). " in ¶42).
As per Claim 4, the combination of Humayun, Ku, and Xu teaches or suggests all limitations of Claim 1. Humayun further discloses wherein the occlusion region is an overlap between a target detection result corresponding to the instance of the one or more flexible objects and an adjacent detection result corresponding to the further instance of the one or more flexible objects (as per "The scene is preferably a ‘dense’ object scene, which can include a plurality of overlapping objects (e.g., where one or more objects are occluded by another object within the scene; the object scene can include a first plurality of objects that partially occludes a second plurality of objects; etc.)." in ¶49, as per "Fig. 2A)
As per Claim 5, the combination of Humayun, Ku, and Xu teaches or suggests all limitations of Claim 4.
Humayun further discloses determining an occlusion state for the occlusion region (as per " the object detector can determine: individual instances of one or more object types, object parameters for each object (e.g., pose, principal axis, occlusion, etc.), total object count, and/or other object information." in ¶33) wherein the occlusion state is (3) an uncertain occlusion state when the overlap between the target detection result and the adjacent detection result is uncertain. (as per "verifying that grasp points with a low probability of success actually result in failed grasps; increasing the prediction confidence for certain predictions, and/or otherwise validating or updating the network." in ¶77). Humayun fails to disclose wherein the occlusion state is one of:
- (1) an adjacent occlusion state representing the adjacent detection result below the target detection result in the occlusion region
- (2) a target occlusion state representing the target detection result below the adjacent detection result in the occlusion region
See Claim 4 for teachings of Ku. Ku further discloses wherein the occlusion state is one of:
- (1) an adjacent occlusion state representing the adjacent detection result below the target detection result in the occlusion region ("In a first embodiment where the occlusion score is for the keypoint, this can include assigning a low score (e.g., “0”) to keypoints of the object that overlap with the mask, and assigning a high score (e.g., “1”) to keypoints of the object that do not overlap with the mask. In a second embodiment where the occlusion score is for the object or subregion, this can include determining the portion of the object instance or subregion that does not overlap with the mask." in ¶59)
- (2) a target occlusion state representing the target detection result below the adjacent detection result in the occlusion region ("In a first embodiment where the occlusion score is for the keypoint, this can include assigning a low score (e.g., “0”) to keypoints of the object that overlap with the mask, and assigning a high score (e.g., “1”) to keypoints of the object that do not overlap with the mask. In a second embodiment where the occlusion score is for the object or subregion, this can include determining the portion of the object instance or subregion that does not overlap with the mask." in ¶59)
In this way, the system operates to grasp objects in occluded environments (¶22). Like Humayun and Xu, Ku is concerned with monitoring objects.
Therefore, from these teachings of Humayun, Ku, and Xu, one of ordinary skill in the art before the effective filing date would have found it obvious to apply the teachings of Humayun and Xu to the system of Ku since doing so would generate detection features and results based on image data. (¶22) Such modification would also allow the system to generate an occlusion score for an object as well as selecting a most visible object face per detected object based on the occlusion scores. (¶17)
As per Claim 6, the combination of Humayun, Ku, and Xu teaches or suggests all limitations of Claim 4. Humayun further discloses determining an occlusion state for the occlusion region based on the detection features corresponding with the target detection result (as per "The object detector functions to detect objects and/or other information in images. For example, the object detector can determine: individual instances of one or more object types, object parameters for each object (e.g., pose, principal axis, occlusion, etc.), total object count, and/or other object information." in ¶33).
Humayun fails to disclose determining an occlusion state for the occlusion region based on the detection features corresponding with the adjacent detection result in the occlusion region.
See Claim 4 for teachings of Ku. Ku further discloses of determining an occlusion state for the occlusion region based on the detection features corresponding with the adjacent detection result in the occlusion region (as per "In a first embodiment where the occlusion score is for the keypoint, this can include assigning a low score (e.g., “0”) to keypoints of the object that overlap with the mask, and assigning a high score (e.g., “1”) to keypoints of the object that do not overlap with the mask. In a second embodiment where the occlusion score is for the object or subregion, this can include determining the portion of the object instance or subregion that does not overlap with the mask." in ¶59).
In this way, the system operates to grasp objects in occluded environments (¶22). Like Humayun and Xu, Ku is concerned with monitoring objects.
Therefore, from these teachings of Humayun, Ku, and Xu, one of ordinary skill in the art before the effective filing date would have found it obvious to apply the teachings of Humayun and Xu to the system of Ku since doing so would generate detection features and results based on image data. (¶22) Such modification would also allow the system to generate an occlusion score for an object as well as selecting a most visible object face per detected object based on the occlusion scores. (¶17)
As per Claim 7, the combination of Humayun, Ku, and Xu teaches or suggests all limitations of Claim 1. Humayun further discloses:
the detection features include edge features, key points, depth values, or a combination thereof (as per "The object parameters can be determined using an object detector (e.g., YOLO, RCN, etc.), which can be trained on synthetic images. The object parameters can be: object keypoints (e.g., keypoints along the object surface, bounding box corners, side centroids, centroid, etc.), object axes (e.g., major axis, minor axis, a characteristic axis, etc.), object pose, surface normal vectors, and/or any other suitable object parameters. " in ¶52, as per "The imaging systems can determine one or more RGB images, depth images (e.g., pixel aligned with the RGB, wherein the RGB image and the depth image can be captured by the same or different sensor sets)." in ¶31);
further comprising: generating the positive identification information for a region in the image data when the edge information, key point information, height measure information, or a combination thereof (as per "The output of the detector can include one or more features, keypoints (e.g., surface keypoints, unique object features, object bounding box features, etc.), labels (e.g., face labels, silhouette labels, texture labels, haptic labels), one or more object masks, one or more scores (e.g., a visibility score for each object, for each feature, etc.), and/or any other suitable information." in ¶36);
generating the detection result includes generating the detection result based on the edge information, key point information, height measure information, or a combination thereof (as per "the object detector can determine: individual instances of one or more object types, object parameters for each object (e.g., pose, principal axis, occlusion, etc.), total object count, and/or other object information." in ¶33, as per "The grasp selector 146 is preferably configured to select grasp points from the output of the graspability network, but can additionally or alternatively be configured to select grasp points from the output of the object detector" in ¶41).
Claims 8 and 15 recite limitations substantially identical to claim 1, differing only in statutory form (system and computer-readable medium). Therefore, the rejection of claim 1 is applied to claims 8 and 15 mutatis mutandis.
Claims 2–7 depend from claim 1. Claims 9, 10, 11–14, 16–20 recite limitations corresponding to claims 2, 3, 4–7, respectively, and are addressed mutatis mutandis as follows: claim 2 ↔ claims 9 and 16; claim 3 ↔ claim 10; claim 4 ↔ claims 11 and 17; claim 5 ↔ claims 12 and 18; claim 6 ↔ claims 13 and 19; claim 7 ↔ claims 14 and 20.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Divakaran (US Pub. No. 20180189573) discloses real-time detection, tracking and occlusion reasoning.
Rodrigues (US Pub. No. 20220281114) discloses performing grip region detection.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TYLER R ROBARGE whose telephone number is (703)756-5872. The examiner can normally be reached Monday - Friday, 8:00 am - 5:00 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ramon Mercado can be reached at (571) 270-5744. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/T.R.R./Examiner, Art Unit 3658
/TRUC M DO/Primary Examiner, Art Unit 3658