DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The Amendment filed February 25th, 2026 has been entered. Claims 1-20 remain pending in the application. Claims 1-3, 5, 8-9, 12, 15-16, and 18 are amended. Claims 4, 6-7, 10-11, 13, 17, and 19-20 have not been amended. Upon further consideration, the Examiner agrees with the Applicant that claims 8, 9, and 13 do not invoke 35. U.S.C. 112(f). The Examiner acknowledges the correction to the part number for the second transformer as shown in Figure 3; the objection to paragraph [0055] of the Specification is withdrawn. The amendments to the Claims have overcome each and every objection and 112(b) rejection previously set forth in the Non-Final Office Action mailed December 4th, 2025.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 4, 6-7, 8, 11, 13-14, 15, and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Kriegler (PrimitivePose: 3D Bounding Box Prediction of Unseen Objects via Synthetic Geometric Primitives, 2023), in view of Forshaw (US 20230237210 A1) and Nvidia Forums A (2D BBox to capture only visible object, 2022).
Regarding Claim 1, Kriegler teaches a method to define a bounding polygon in a view of a three-dimensional scene (Figure 3 demonstrates defining bounding polygons in a view of a three-dimensional scene), the method comprising:
defining a plurality of rays with each ray extending from a common viewpoint of a virtual three-dimensional model to a respective vertex in a plurality of vertices of a virtual three- dimensional representation of an object of interest in the virtual three-dimensional model (Section A. Generating Synthetic Data for Object Recognition: “a ray-tracing algorithm sends rays from the camera focal point through every pixel of the virtual image plane and stores all intersections of the rays with object meshes”. Notes: The camera is a common viewpoint; the object of interest is necessarily intersected via ray tracing); and
defining, in an image of the virtual three-dimensional model that is created from the viewpoint, the bounding polygon for the object of interest that encompasses vertices of the object of interest that are visible.
Kriegler does not explicitly teach determining a set of occluded rays with respect to the object of interest in the plurality of rays, where each ray in the set of occluded rays intercepts a respective occluding object in the virtual three-dimensional model prior to reaching the respective vertex of the object of interest when extending from the viewpoint, and defining a set of visible rays with respect to the object of interest within the plurality of rays that excludes the occluded set of rays.
However, it is heavily implicit that Kriegler teaches determining a set of occluded rays with respect to the object of interest in the plurality of rays, where each ray in the set of occluded rays intercepts a respective occluding object in the virtual three-dimensional model prior to reaching the respective vertex of the object of interest when extending from the viewpoint (Section A. Generating Synthetic Data for Object Recognition: “For the calculation of object occlusion, a ray-tracing algorithm sends rays from the camera focal point through every pixel of the virtual image plane and stores all intersections of the rays with object meshes”; Section A. Generating Synthetic Data for Object Recognition: “Finally, we use the occlusion measure calculated automatically during annotation to filter all objects with occlusion greater than 90% for the evaluation”. Notes: since occlusion percentage of a number of rays associated with a target object is known, the number of visible and occluded rays is also inherently known for an object of interest. The object of interest is the object in which occlusion is calculated for. An intersection between the ray and an object mesh is necessarily a coordinate, and hence a vertex of the object of interest on the object mesh); and
defining a set of visible rays with respect to the object of interest within the plurality of rays that excludes the occluded set of rays (“Section A. Generating Synthetic Data for Object Recognition: “Finally, we use the occlusion measure calculated automatically during annotation to filter all objects with occlusion greater than 90% for the evaluation”. Notes: since occlusion percentage of a number of rays associated with a target object is known, the number of visible and occluded rays is also inherently known)
The use of projected rays for determining object occlusion in terms of a proportion of visible and occluded rays is well known in the art, as is demonstrated by Forshaw.
Forshaw teaches determining a set of occluded rays with respect to the object of interest in the plurality of rays, where each ray in the set of occluded rays intercepts a respective occluding object in the virtual three-dimensional model prior to reaching the respective vertex of the object of interest when extending from the viewpoint (Paragraph [0032]: “In the described embodiments, the occlusion metric is a numerical value indicating the number of occluded rays in proportion to the total number of rays that intersect the collision surface of the target object (this could, for example, be the number of occluded rays in proportion to the total number of rays that intersect the target object, or the number of non-occluded rays in proportion to the total).”. Notes: Occluded rays are necessarily included in the number of rays that intersect the collision surface, as rays note intersections for all objects intercepted by the ray); and
defining a set of visible rays with respect to the object of interest within the plurality of rays that excludes the occluded set of rays (Paragraph [0032]: “In the described embodiments, the occlusion metric is a numerical value indicating the number of occluded rays in proportion to the total number of rays that intersect the collision surface of the target object (this could, for example, be the number of occluded rays in proportion to the total number of rays that intersect the target object, or the number of non-occluded rays in proportion to the total).” Notes: non-occluded rays are visible rays);
Kriegler and Forshaw are considered analogous in the art with respect to the use of rays for determining object occlusion. One having ordinary skill in the art would appreciate that Forshaw explicitly states what is heavily implicit in Kriegler regarding occlusion of an object determined via a set of occluded rays, as well as determining visible rays.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the method of defining a bounding polygon with regards to object occlusion and visibility of Kriegler with the method of determining occlusion via a set of occluded rays and determining of visible rays of Forshaw; Doing so would yield the predictable result of determining occlusion metrics reliably.
Kriegler does not explicitly teach defining visible rays that intercept vertices of an object of interest.
However, as established by Forshaw, establishing visible rays in regards to occlusion is well known. It would have been obvious to a person having ordinary skill in the art that a bounding polygon identifying an object of interest encompasses all visible portions of the object of interest, which necessarily includes all vertices of the object intercepted by visible rays.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention that the bounding polygon for identifying an object of interest of Kriegler as modified inherently includes vertices of the object of interest intercepted by a set of visible rays.
Kriegler as modified does not teach that the bounding polygon excludes at least one vertex intercepted by a respective ray in the set of occluded rays.
However, Nvidia Forums teaches the bounding polygon excluding at least one vertex intercepted by a respective ray in the set of occluded rays (Question by MontyPy: “Is there anyway that we can have the 2d bboxes capture only the visible portion of an object and not occluded portions? The 2d tight bbox still captures some occluded portions. I’m trying to label only the small black band at the top”; Answer by pcallender: “2dBBoxes are computed by the renderer, by ray-tracing the scene and computing the actual 2dbbox of the visible pixels for every object. After that there is a post-processing step which merges the 2dbboxes of the objects sharing the same semantic entity”. Notes: Occluded portions are excluded in the method of calculating the bounding polygon by only considering visible pixels via ray-tracing. The bounding polygon in question (see image) excludes some portions of the of the object that are occluded (as determined via ray-tracing)).
Kriegler as modified and Nvidia Forums are considered analogous in the art with respect to forming bounding polygons for object detection. One would be motivated to only form bounding polygons around visible portions of objects of interest, and exclude occluded portions of objects of interest, to reduce visual clutter for a viewer.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the bounding polygon method of Kriegler as modified with the bounding polygon method that excludes occluded portions of objects of interest of Nvidia Forums; Doing so would yield the predictable result of only forming bounding polygons around visible portions of an object of interest, reducing visual clutter.
Regarding Claim 4, the method of Claim 1 is rejected over Kriegler as modified.
Kriegler as modified teaches the bounding polygon comprising more than four sides (Kriegler, Figure 3 clearly illustrates formed bounding polygons with more than four sides).
Regarding Claim 6, the method of Claim 1 is rejected over Kriegler as modified.
Kriegler as modified teaches the method of Claim 1, further comprising determining that a number of vertices that are respective destinations of rays in the set of occluded rays is below a threshold, and wherein defining the bounding polygon for the object of interest is based on determining that the number of vertices that are respective destinations of rays in the set of occluded rays is below the threshold (Kriegler, Section A. Generating Synthetic Data for Object Recognition: “For the calculation of object occlusion, a ray-tracing algorithm sends rays from the camera focal point through every pixel of the virtual image plane and stores all intersections of the rays with object meshes”; Kriegler, Section A. Generating Synthetic Data for Object Recognition: “Finally, we use the occlusion measure calculated automatically during annotation to filter all objects with occlusion greater than 90% for the evaluation”; Forshaw, Paragraph [0032]: “In the described embodiments, the occlusion metric is a numerical value indicating the number of occluded rays in proportion to the total number of rays that intersect the collision surface of the target object (this could, for example, be the number of occluded rays in proportion to the total number of rays that intersect the target object, or the number of non-occluded rays in proportion to the total)”. Notes: each object has an associated set of rays that reach a vertex in its mesh. If over 90% of the set of rays is an occluded ray (intersects with another object mesh beforehand), then the object is filtered (not considered for detection), and subsequently, does not have a bounding polygon generated for it).
Regarding Claim 7, the method of Claim 6 is rejected over Kriegler as modified.
Kriegler as modified teaches the threshold is based on a number of rays in the plurality of rays (Kriegler, Section A. Generating Synthetic Data for Object Recognition: “a ray-tracing algorithm sends rays from the camera focal point through every pixel of the virtual image plane and stores all intersections of the rays with object meshes”; Kriegler, Section A. Generating Synthetic Data for Object Recognition: “Finally, we use the occlusion measure calculated automatically during annotation to filter all objects with occlusion greater than 90% for the evaluation”; Forshaw, Paragraph [0032]: “In the described embodiments, the occlusion metric is a numerical value indicating the number of occluded rays in proportion to the total number of rays that intersect the collision surface of the target object (this could, for example, be the number of occluded rays in proportion to the total number of rays that intersect the target object, or the number of non-occluded rays in proportion to the total)”. Notes: each object has an associated set of rays that reach a vertex in its mesh. The threshold is based on a percentage of the occluded rays of the plurality of rays, where if over 90% of the plurality of rays is an occluded ray (intersects with another object mesh beforehand), then the object is filtered (not considered for detection), and subsequently, does not have a bounding polygon generated for it).
Claim 8, being similar in scope to Claim 1, is rejected under the same rationale.
Claim 11, being similar in scope to Claim 4, is rejected under the same rationale.
Claim 13, being similar in scope to Claim 6, is rejected under the same rationale.
Claim 14, being similar in scope to Claim 7, is rejected under the same rationale.
Claim 15, being similar in scope to Claim 1, is rejected under the same rationale.
Claim 19, being similar in scope to Claim 6, is rejected under the same rationale.
Claim 20, being similar in scope to Claim 7 is rejected under the same rationale.
Claims 2, 5, 9, 12, 16 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Kriegler (PrimitivePose: 3D Bounding Box Prediction of Unseen Objects via Synthetic Geometric Primitives, 2023), in view of Forshaw (US 20230237210 A1) and Nvidia Forums A (2D BBox to capture only visible object, 2022), in further view of Nvidia Forums B (Occluded objects-bounding boxes, 2022).
Regarding Claim 2, the method of Claim 1 is rejected over Kriegler as modified.
Kriegler as modified teaches the method of Claim 1, further comprising:
Defining a second plurality of rays with each ray extending from the viewpoint to a respective vertex in a second plurality of vertices of a virtual three-dimensional representation of the respective occluding object (Kriegler, Section A. Generating Synthetic Data for Object Recognition: “a ray-tracing algorithm sends rays from the camera focal point through every pixel of the virtual image plane and stores all intersections of the rays with object meshes”; Kriegler, Section A. Generating Synthetic Data for Object Recognition: “Finally, we use the occlusion measure calculated automatically during annotation to filter all objects with occlusion greater than 90% for the evaluation”; Forshaw, Paragraph [0032]: “In the described embodiments, the occlusion metric is a numerical value indicating the number of occluded rays in proportion to the total number of rays that intersect the collision surface of the target object (this could, for example, be the number of occluded rays in proportion to the total number of rays that intersect the target object, or the number of non-occluded rays in proportion to the total)”. Notes: since rays are sent to every pixel of the virtual image plane, and stores all intersections of the rays with object meshes, the rays that are occluded rays (pass through multiple object meshes) are also defined, where the intersection between the occluding object mesh of the occluding object and the ray is necessarily a vertex on the occluding object mesh); and
determining, based on the set of visible rays and the second plurality of rays, an edge dividing an image of the respective occluding object and an image of the object of interest within the image of the virtual three-dimensional model (Nvidia Forums A, Question by MontyPy: “Is there anyway that we can have the 2d bboxes capture only the visible portion of an object and not occluded portions? The 2d tight bbox still captures some occluded portions. I’m trying to label only the small black band at the top”; Nvidia Forums A, Answer by pcallender: “2dBBoxes are computed by the renderer, by ray-tracing the scene and computing the actual 2dbbox of the visible pixels for every object. After that there is a post-processing step which merges the 2dbboxes of the objects sharing the same semantic entity”. Notes: Occluded portions are excluded in the method of calculating the bounding polygon by only considering visible pixels via ray-tracing. Because the scene is ray-traced to determine visible pixels for every object, edges are determined inherently when determining whether a pixel is visible or not from the viewpoint. Since the bounding polygons are formed with respect to visible pixels, the bounding polygon necessarily comprises a side defined by the edge, where the edge, in its broadest reasonable interpretation, is the outline of visible pixels for an object).
Kriegler as modified does not explicitly teach that the bounding polygon comprises a side defined by the edge of an occluding object and an object of interest.
However, Nvidia Forums B teaches the bounding polygon comprising a side defined by the edge of an occluding object and an object of interest (Question by Andrew.hannel: “When using Isaacsim Replicator Composer- bounding boxes (both tight & loose) are generated for all objects, even if they are completely occluded (hidden behind other objects). How can I generate bounding boxes just for the portion of objects that are visible ??”; Answer by pcallender: “Hi Andrew. I checked into this. Here’s how this should work: 1. Loose will always be annotated, regardless of occlusions. 2. Tight should always be cropped, or removed if fully occluded. This should be what you want”. Notes: tight bounding boxes are fit to visible portions of an object of interest by ray-tracing, as specified by Nvidia Forums A. As seen in the image, edges of occluding objects and an object of interest are utilized as reference for a bounding polygon. In particular, light blue excavator is partially occluded by the dark blue excavator; the right side of the bounding polygon identifying the excavator is defined by the edge of the occluding object (arm portion of the dark blue excavator) and the visible portion of the object of interest (the back end of the light blue excavator is occluded).
Kriegler as modified and Nvidia Forums B are considered analogous in the art with respect to bounding boxes that factor in occluding objects when identifying objects of interest that are partially visible. Nvidia Forums A (Kriegler as modified) and Nvidia Forums B are centered around Isaac Sim, which can display bounding polygons around objects of interest. A person having ordinary skill in the art would appreciate that Nvidia Forums A (Kriegler as modified) and Nvidia Forums B address how bounding polygons are formed for visible portions of an object of interest with respect to the occluding object, and that image of Nvidia Forums B illustrates the process of forming bounding polygons for partially occluded objects of interest as specified in both Nvidia Forums A (Kriegler as modified) and Nvidia Forums B.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the formation of bounding polygon for an object of interest with respect to an occluding object Kriegler as modified with the bounding polygon comprising a side defined by an edge of an occluding object and the object of interest of Nvidia Forums B; Doing so would yield the predictable result of bounding polygons for objects of interest that are at least partially defined by an edge between the objects of interest and their respective occluding objects.
Regarding Claim 5, the method of Claim 1 is rejected over Kriegler as modified.
Kriegler as modified does not explicitly teach the bounding polygon comprises at least one side corresponding to a vertex of the respective occluding object in the virtual three- dimensional model.
However, Nvidia Forums B teaches the bounding polygon comprises at least one side corresponding to a vertex of the respective occluding object in the virtual three- dimensional model (Question by Andrew.hannel: “When using Isaacsim Replicator Composer- bounding boxes (both tight & loose) are generated for all objects, even if they are completely occluded (hidden behind other objects). How can I generate bounding boxes just for the portion of objects that are visible ??”; Answer by pcallender: “Hi Andrew. I checked into this. Here’s how this should work: 1. Loose will always be annotated, regardless of occlusions. 2. Tight should always be cropped, or removed if fully occluded. This should be what you want”. Notes: tight bounding boxes are fit to visible portions of an object of interest by ray-tracing, as specified by Nvidia Forums A (Kriegler as modified). As seen in the image, edges of occluding objects and an object of interest are utilized as reference for a bounding polygon. In particular, light blue excavator is partially occluded by the dark blue excavator; the right side of the bounding polygon identifying the excavator is defined by the edge (which includes a vertex) of the occluding object (arm portion of the dark blue excavator) and the visible portion of the object of interest (the back end of the light blue excavator is occluded)
Kriegler as modified and Nvidia Forums B are considered analogous in the art with respect to bounding boxes that factor in occluding objects when identifying objects of interest that are partially visible. Nvidia Forums A (Kriegler as modified) and Nvidia Forums B are centered around Isaac Sim, which can display bounding polygons around objects of interest. A person having ordinary skill in the art would appreciate that Nvidia Forums A (Kriegler as modified) and Nvidia Forums B address how bounding polygons are formed for visible portions of an object of interest with respect to the occluding object, and that image of Nvidia Forums B illustrates the process of forming bounding polygons for partially occluded objects of interest as specified in both Nvidia Forums A (Kriegler as modified) and Nvidia Forums B.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the formation of bounding polygon for an object of interest with respect to an occluding object Kriegler as modified with the bounding polygon comprising a side defined by a vertex of an occluding object; Doing so would yield the predictable result of bounding polygons for objects of interest that are at least partially defined by a vertex of an occluding object.
Claim 9, being similar in scope to Claim 2, is rejected under the same rationale.
Claim 12, being similar in scope to Claim 5, is rejected under the same rationale.
Claim 16, being similar in scope to Claim 2, is rejected under the same rationale.
Claim 18, being similar in scope to Claim 5, is rejected under the same rationale.
Claims 3, 10, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Kriegler (PrimitivePose: 3D Bounding Box Prediction of Unseen Objects via Synthetic Geometric Primitives, 2023), in view of Forshaw (US 20230237210 A1), Nvidia Forums A (2D BBox to capture only visible object, 2022) and Nvidia Forums B (Occluded objects-bounding boxes, 2022), in further view of Lehner (3D-VField: Adversarial Augmentation of Point Clouds for Domain Generalization in 3D Object Detection, 2022).
Regarding Claim 3, the method of Claim 1 is rejected over Kriegler as modified.
Kriegler as modified does not teach that the virtual three-dimensional representation of the object of interest comprises a representation of a damaged version of the object of interest.
However, Lehner teaches that the virtual three-dimensional representation of the object of interest comprises a representation of a damaged version of the object of interest (Abstract: “Our approach constrains 3D points to slide along their sensor view rays while neither adding nor removing any of them. The obtained vectors are transferable, sample-independent and preserve shape and occlusions. Despite training only on a standard dataset, such as KITTI, augmenting with our vector fields significantly improves the generalization to differently shaped objects and scenes”; Conclusion: “In this paper we presented 3D-VField: an adversarial augmentation method for point clouds to improve the object detection performance on natural adversarial examples and out-of-domain data, such as rare, damaged cars, or vehicles from different regions”; Figure 1 illustrates how a bounding polygon are formed around a damaged car (bumper is damaged)).
Kriegler as modified and Lehner are considered analogous in the art with respect to object detection with bounding polygons. A common motivation in the art is to generalize object detection so that different variants can be detected as belonging to a particular object class, as is evident in Lehner, with the detection of vehicles that may be damaged.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the detection of an object of interest and forming of a bounding polygon of Kriegler as modified with the detection of damaged variants of an object of interest and subsequent formation of bounding polygons of Lehner; Doing so would yield the predictable result of a more generalized approach for detecting and forming bounding polygons for objects of interest that may have different variants, such as damaged variants.
Claim 10, being similar in scope to Claim 3, is rejected under the same rationale.
Claim 17, being similar in scope to Claim 3, is rejected under the same rationale.
Response to Arguments
Applicant’s arguments with respect to claims 1, 3, 8, 10, 15 and 17 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RAYMOND CHUN LAM LI whose telephone number is (571)272-5124. The examiner can normally be reached M-F 8:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at 571-272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RAYMOND CHUN LAM LI/Examiner, Art Unit 2614
/KENT W CHANG/Supervisory Patent Examiner, Art Unit 2614
Prosecution Insights