DETAILED ACTION
Claims 1-20 are currently pending.
The previous objection to the title of the invention is withdrawn due to Applicant’s amendment.
Response to Arguments
Applicant's arguments filed 6/23/26 have been fully considered but they are not persuasive.
The Applicant argues on pages 10-11 of the response in essence that: Amended Claim 1 recites a novel and critical combination of limitations that Peake neither discloses nor suggests: "extract a unique color value of the at least one object from the virtual map environment corresponding to the adjusted viewpoint; and generate ground truth (GT) data for the image data based on the unique color value." Peake merely discloses generating a static "label layer" mapped to a "normal layer," where pixels are encoded with a first channel set representing predefined, text-based or state- based classification parameters (Peake, 1 [0176]).
Peake discloses generating virtual or simulated information including graphic-based information, e.g., pixel information, RGB information (paragraph 46). Based on the generated virtual information, feature training datasets are generated (paragraph 173).
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
Use of the word “means” (or “step for”) in a claim with functional language creates a rebuttable presumption that the claim element is to be treated in accordance with 35 U.S.C. 112(f) (pre-AIA 35 U.S.C. 112, sixth paragraph). The presumption that 35 U.S.C. 112(f) (pre-AIA 35 U.S.C. 112, sixth paragraph) is invoked is rebutted when the function is recited with sufficient structure, material, or acts within the claim itself to entirely perform the recited function.
Absence of the word “means” (or “step for”) in a claim creates a rebuttable presumption that the claim element is not to be treated in accordance with 35 U.S.C. 112(f) (pre-AIA 35 U.S.C. 112, sixth paragraph). The presumption that 35 U.S.C. 112(f) (pre-AIA 35 U.S.C. 112, sixth paragraph) is not invoked is rebutted when the claim element recites function but fails to recite sufficiently definite structure, material or acts to perform that function.
Claim elements in this application that use the word “means” (or “step for”) are presumed to invoke 35 U.S.C. 112(f) except as otherwise indicated in an Office action. Similarly, claim elements that do not use the word “means” (or “step for”) are presumed not to invoke 35 U.S.C. 112(f) except as otherwise indicated in an Office action.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitations use a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are:
a first virtual sensor, and
a second virtual sensor in claims 4 and 11.
Because these claim limitations are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have these limitations interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitations to avoid them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitations recite sufficient structure to perform the claimed function so as to avoid them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 4, 5, 11 and 12 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
The claim limitations “a first virtual sensor” and “a second virtual sensor” listed above invokes 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. However, the written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed function and to clearly link the structure, material, or acts to the function. The Applicant's specification does not provide an association between the structure and the function. Therefore, claims 4, 5, 11 and 12 are indefinite and are rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph.
Applicant may:
(a) Amend the claim so that the claim limitation will no longer be interpreted as a limitation under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph;
(b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(c) Amend the written description of the specification such that it clearly links the structure, material, or acts disclosed therein to the function recited in the claim, without introducing any new matter (35 U.S.C. 132(a)).
If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the function so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function, applicant should clarify the record by either:
(a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed function and clearly links or associates the structure, material, or acts to the claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed function. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 2, 4, 5, 7-9, 11, 12, 14-17 and 20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Peake et al. US Publication 2020/0074266 (hereafter “Peake”).
Referring to claims 1, 8 and 16, Peake discloses apparatus for generating training data for a neural network, the apparatus comprising:
a memory configured to store one or more instructions; and
a processor configured to execute the one or more instructions stored in the memory, wherein the instructions, when executed by the processor, cause the processor to:
configure a virtual map environment including a road and at least one object moving on the road (paragraph 61, In various embodiments, the imaging engine 102 may be a virtual engine or gaming engine (e.g., a DirectX-based, OpenGL-based, or other gaming engine) that can render 2D and/or 3D images of a virtual environment. The virtual environment, as referred to herein, may include a computer rendered environment including streets, roads, intersections, overpasses, vehicles, pedestrians, buildings or other structures, traffic lights or signs, or any other object or surface capable of being rendered in a virtual environment, such as a 2D or 3D environment);
set a sensing environment of a virtual sensor system within the virtual map environment, wherein setting the sensing environment comprises adjusting a viewpoint of the virtual sensor system to capture the at least one object from various viewpoints (paragraph 63, A photo-realistic scene, such as photo-realistic scene 200 of FIG. 2A, may comprise a two-dimensional (2D) image that simulates a real-world scene as captured by a real-world 2D camera or other sensor);
generate image data representing the virtual map environment from the adjusted viewpoint using the virtual sensor system (paragraph 170, Method 900 begins (902) at block 904 where, e.g., automated training dataset generator 100, generates a plurality of imaging scenes (e.g., scenes 390, 400 and/or 450) defining a virtual environment (e.g., the virtual environments of FIGS. 3, 4A, and 4B, respectively). The plurality of imaging scenes may include a plurality of photo-realistic scenes (e.g., as exemplified by scenes 400 and 450) and a plurality of corresponding depth-map-realistic scenes (e.g., as exemplified by scene 390));
extract a unique color value of the at least one object from the virtual map environment corresponding to the adjusted viewpoint (paragraph 46, Such virtual or simulated information may include, but is not limited to graphic-based information, e.g., pixel information, RGB information);
generate ground truth (GT) data for the image data based on the unique color value (paragraph 173, At block 910, method 900 may further include generating one or more feature training datasets based on at least one of (i) the plurality of photo-realistic scenes (e.g., scenes 400 and 450), (ii) the plurality of depth-map-realistic scenes (e.g., scene 390), or (iii) the environment-object data (e.g., data associated with objects and surfaces 391-398 of FIG. 3, objects and surfaces 401-418 of FIG. 4A, and/or objects and surfaces 451-482)); and
generate the training data by combining the image data and the GT data (paragraph 173, As described herein, feature training dataset may be associated with training a machine learning model to control an autonomous vehicle (e.g., vehicle 700 or 760) in a real-world autonomous driving application).
Referring to claims 2, 9 and 17, Peake discloses wherein the processor is configured to set a time condition including daytime or nighttime and a weather condition in the virtual map environment (paragraph 77, parameter setting control 422 may be used to set the date (e.g., Jun. 1, 2001), time (e.g., 4:20 PM), traffic condition (e.g., no traffic, light traffic, high traffic, etc.), temperature (e.g., 93 degrees), sun condition (e.g., bright), and/or contrast of the scene 400. Other parameter setting controls include weather parameter setting control 424, which may be used to configure the weather conditions (e.g., clear, overcast, partly cloudy, raining, hailing, snowing, etc.) of the virtual environment).
Referring to claims 4 and 11, Peake discloses wherein the virtual sensor system comprises:
a first virtual sensor configured to acquire a color image (paragraph 39, The sensor simulator may simulate lidar (e.g., light detection and ranging) readings using ray casting or depth maps, for example, and/or images captured by a camera, etc); and
a second virtual sensor configured to acquire an instance segmentation image (paragraph 114, The segmentation module 510 is generally configured to identify distinct objects within the sensor data representing the sensed environment).
Referring to claims 5 and 12, Peake discloses wherein the processor is configured to further extract contour information of the at least one object using the second virtual sensor (paragraph 114, The segmentation module 510 is generally configured to identify distinct objects within the sensor data representing the sensed environment).
Referring to claim 7, Peake discloses wherein the virtual sensor system includes a plurality of virtual sensors installed at a plurality of different positions on the virtual map environment (paragraph 39, one or more virtual sensors may be placed in various positions around one or more vehicles in the virtual environment for the purpose of generating the simulated sensor data), and
wherein the processor is configured to track a position of the at least one object moving through the virtual map environment based on a unique identifier that corresponds to the unique color value (paragraph 88, each of the objects or surfaces may be associated with a tracking identifier (TID) (e.g., a unique identifier (ID)) that tracks objects and surfaces (e.g., vehicles) within each frame).
Referring to claims 14 and 20, Peake discloses wherein the virtual sensor system includes a plurality of virtual sensors installed at a plurality of different positions on the virtual map environment (paragraph 39, one or more virtual sensors may be placed in various positions around one or more vehicles in the virtual environment for the purpose of generating the simulated sensor data).
Referring to claim 15, Peake discloses tracking a position of the at least one object moving through the virtual map environment based on a unique identifier corresponding to the unique color value (paragraph 88, each of the objects or surfaces may be associated with a tracking identifier (TID) (e.g., a unique identifier (ID)) that tracks objects and surfaces (e.g., vehicles) within each frame).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 3, 10 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Peake et al. US Publication 2020/0074266 as applied to claims 1, 8 and 16 above, and further in view of Smolyanskiy et al. US Publication 2022/0415059 (hereafter “Smolyanskiy”).
Referring to claims 3, 10 and 18, Peake discloses wherein the viewpoint adjusted to capture the at least one object includes includes an installation position and height of the virtual sensor system (paragraph 99, the sensor simulator 104 may position one or more virtual sensors in a virtual environment (e.g., any of the virtual environment(s) depicted and described for FIGS. 2A, 2B, 3, 4A, and/or 4B)) (paragraph 88, a descriptor of each object or surface may include a position value of the object within the virtual environment and an orientation of the object within the virtual environment).
Peake does not disclose expressly wherein the photographing environment includes the rotation angle.
Smolyanskiy discloses wherein the viewpoint adjusted to capture the at least one object includes an installation position, height, and rotation angle of the virtual sensor system (paragraph 124, for each pixel contained with an annotation, the annotation may be used to compute corresponding location, geometry, and/or orientation information (e.g., where the object is located—such as the object center—relative to each pixel, object height, object width, object orientation (e.g., rotation angles relative to the orientation of the projection image), and/or the like)).
Before the effective filing date of the claimed invention, it would have obvious to a person of ordinary skill in the art to provide a rotation angle within the 3D graphic road environment. The motivation for doing so would have been to improve labeling of data to better understand that data. Therefore, it would have been obvious to combine Choe with Peake to obtain the invention as specified in claims 3, 10 and 18.
Claims 6, 13 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Peake et al. US Publication 2020/0074266 as applied to claims 1, 8 and 16 above, and further in view of Choe et al. US Publication 2024/0001957 (hereafter “Choe”).
Referring to claims 6, 13 and 19, Peake discloses the GT data, but does not disclose expressly wherein the training GT data includes at least one of bounding box information indicating an area in which there is the at least one object and mask information indicating an identifier for identifying the at least one object.
Choe discloses wherein the GT data includes at least one of bounding box information indicating a position of the at least one object and mask information used for classification of the at least one object (paragraph 26, A segmentation mask corresponding to each instance of an object may also be generated and used to determine a bounding shape that encompasses each instance).
Before the effective filing date of the claimed invention, it would have obvious to a person of ordinary skill in the art to include bounding box and mask information for an object. The motivation for doing so would have been to improve data organization to better isolate objects for processing, and to ultimately help determine a path for autonomous vehicles. Therefore, it would have been obvious to combine Choe with Peake to obtain the invention as specified in claims 6, 13 and 19.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PETER K HUNTSINGER whose telephone number is (571)272-7435. The examiner can normally be reached Monday - Friday 8:30 - 5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Benny Q Tieu can be reached at 571-272-7490. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PETER K HUNTSINGER/Primary Examiner, Art Unit 2682
Prosecution Insights